From tim.hochberg at ieee.org Wed Dec 1 06:35:30 2004 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Wed Dec 1 06:35:39 2004 Subject: [Python-Dev] Roster Deadline Message-ID: <41AD5822.2000501@ieee.org> Hi Larry, FYI: I asked EB about the roster deadline and she says that she doesn't know when it is either. Checking on the Lei Out web page didn't help much either. So, you are no wiser now than at the start of this message. -tim From fdrake at acm.org Wed Dec 1 07:34:51 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Dec 1 07:35:12 2004 Subject: [Python-Dev] Python.org current docs In-Reply-To: <200411301612.37180.fdrake@acm.org> References: <8y8jrufo.fsf@python.net> <200411301612.37180.fdrake@acm.org> Message-ID: <200412010134.51160.fdrake@acm.org> On Tuesday 30 November 2004 02:46 pm, Thomas Heller wrote: > http://www.python.org/doc/current/ > and > http://docs.python.org/ > > > still point to 2.3.4 docs. I think everything is properly updated now. Please let me know if I've missed anything. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> From mal at egenix.com Wed Dec 1 10:23:16 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Dec 1 10:23:12 2004 Subject: [Python-Dev] File encodings In-Reply-To: <20041130130934.GC6610@burma.localdomain> References: <20041129190448.GA23399@burma.localdomain> <41AC34D2.1060909@egenix.com> <20041130130934.GC6610@burma.localdomain> Message-ID: <41AD8D84.7070903@egenix.com> Gustavo Niemeyer wrote: > [...] > >>The idiom presented by Bob is the right way to go: wrap >>sys.stdout with a StreamWriter. > > I don't see that as a good solution, since every Python software > that is internationalizaed will have do figure out this wrapping, > introducing extra overhead unnecessarily. I don't see any unnecessary overhead and using the wrappers is really easy, e.g.: # # Application uses Latin-1 for I/O, terminal uses UTF-8 # import codecs, sys # Make stdout translate Latin-1 output into UTF-8 output sys.stdout = codecs.EncodedFile(sys.stdout, 'latin-1', 'utf-8') # Have stdin translate Latin-1 input into UTF-8 input sys.stdin = codecs.EncodedFile(sys.stdin, 'utf-8', 'latin-1') We should probably extend the support in StreamRecoder (which is used by the above EncodedFile helper) to also support Unicode input to .write() and have a special codec 'unicode' that converts Unicode to Unicode, so that you can request the EncodedFile object to return Unicode for .read(). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 01 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From kiko at async.com.br Wed Dec 1 14:08:25 2004 From: kiko at async.com.br (Christian Robottom Reis) Date: Wed Dec 1 14:08:34 2004 Subject: [Python-Dev] Difflib modifications [reposted] Message-ID: <20041201130825.GQ4847@async.com.br> [Reposted to python-dev!] Hello there, We've has done some customizations to difflib to make it work well with pagetests we are running on a project at Canonical, and we are looking for some guidance as to what's the best way to do them. There are some tricky bits that have to do with how the class inheritance is put together, and since we would want to avoid duplicating difflib I figured we'd ask and see if some grand ideas come up. A [rough first cut of the] patch is inlined below. Essentially, it does: - Implements a custom Differ.fancy_compare function that supports ellipsis and omits equal content - Hacks _fancy_replace to skip ellipsis as well. - Hacks best_ratio and cutoff. I'm a bit fuzzy on why this was changed, to be honest, and Celso's travelling today, but IIRC it had to do with how difflib grouped changes. Essentially, what we aim for is: - Ignoring ellipsisized(!) content - Omitting content which is equal I initially thought the best way to do this would be to inherit from SequenceMatcher and make it not return opcodes for ellipsis. However, there is no easy way to replace the class short of rewriting major bits of Differ. I suspect this could be easily changed to use a class attribute that we could override, but let me know what you think of the whole thing. --- /usr/lib/python2.3/difflib.py 2004-11-18 20:05:38.720109040 -0200 +++ difflib.py 2004-11-18 20:24:06.731665680 -0200 @@ -885,6 +885,45 @@ for line in g: yield line + def fancy_compare(self, a, b): + """ + >>> import difflib + >>> engine = difflib.Differ() + >>> got = ['World is Cruel', 'Dudes are Cool'] + >>> want = ['World ... Cruel', 'Dudes ... Cool'] + >>> list(engine.fancy_compare(want, got)) + [] + + """ + cruncher = SequenceMatcher(self.linejunk, a, b) + for tag, alo, ahi, blo, bhi in cruncher.get_opcodes(): + + if tag == 'replace': + ## replace single line + if a[alo:ahi][0].rstrip() == '...' and ((ahi - alo) == 1): + g = None + ## two lines replaced + elif a[alo:ahi][0].rstrip() == '...' and ((ahi - alo) > 1): + g = self._fancy_replace(a, (ahi - 1), ahi, + b, (bhi - 1), bhi) + ## common + else: + g = self._fancy_replace(a, alo, ahi, b, blo, bhi) + elif tag == 'delete': + g = self._dump('-', a, alo, ahi) + elif tag == 'insert': + g = self._dump('+', b, blo, bhi) + elif tag == 'equal': + # do not show anything + g = None + else: + raise ValueError, 'unknown tag ' + `tag` + + if g: + for line in g: + yield line + + def _dump(self, tag, x, lo, hi): """Generate comparison results for a same-tagged range.""" for i in xrange(lo, hi): @@ -926,7 +965,13 @@ # don't synch up unless the lines have a similarity score of at # least cutoff; best_ratio tracks the best score seen so far - best_ratio, cutoff = 0.74, 0.75 + #best_ratio, cutoff = 0.74, 0.75 + + ## reduce the cutoff to have enough similarity + ## between '<something> ... <something>' and '<a> blabla </a>' + ## for example + best_ratio, cutoff = 0.009, 0.01 + cruncher = SequenceMatcher(self.charjunk) eqi, eqj = None, None # 1st indices of equal lines (if any) @@ -981,7 +1026,11 @@ cruncher.set_seqs(aelt, belt) for tag, ai1, ai2, bj1, bj2 in cruncher.get_opcodes(): la, lb = ai2 - ai1, bj2 - bj1 - if tag == 'replace': + + if aelt[ai1:ai2] == '...': + return + + if tag == 'replace': atags += '^' * la btags += '^' * lb elif tag == 'delete': Take care, -- Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3361 2331 From anthony at interlink.com.au Wed Dec 1 14:32:07 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Dec 1 14:33:13 2004 Subject: [Python-Dev] Re: Small subprocess patch In-Reply-To: <Pine.GSO.4.51L2.0411302255440.21491@koeberg.lysator.liu.se> References: <Pine.GSO.4.51L2.0411302123220.18754@koeberg.lysator.liu.se> <Pine.GSO.4.51L2.0411302255440.21491@koeberg.lysator.liu.se> Message-ID: <200412020032.07794.anthony@interlink.com.au> On Wednesday 01 December 2004 09:00, Peter Astrand wrote: > I'm also wondering if patch 1071755 and 1071764 should go into > release24-maint: > > * 1071755 makes subprocess raise TypeError if Popen is called with a > bufsize that is not an integer. Since this isn't changing anything that's user facing (just making the error handling more explicit) this is suitable for the maint branch. > * 1071764 adds a new, small utility function. Please read PEP 6. Maintenance branches are not the place for new features. For an example why, consult almost any code that requires Python 2.2 or later. Chances are you'll find, all over the place, code like: try: True, False except NameError: True, False = 1, 0 From mal at egenix.com Wed Dec 1 16:10:10 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Dec 1 16:10:15 2004 Subject: [Python-Dev] MS VC compiler versions Message-ID: <41ADDED2.30707@egenix.com> Preparing for the distutils patch to allow building extensions using the .NET SDK compilers, I am compiling a list of version numbers and MS compiler logo outputs in order to use these to identify the correct compiler to use for the extensions. These are the compilers I have found so far: * MS VC6 (German version; optimizing VC6 compiler): Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, fuer x86 Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten. * MS .NET SDK 1.1 Compiler (German version; non-optimizing VC7.1 compiler): Microsoft (R) 32-Bit C/C++-Standardcompiler Version 13.10.3077 f?r 80x86 Copyright (C) Microsoft Corporation 1984-2002. Alle Rechte vorbehalten. * MS VC7.1 (aka .NET 2003, US version; optimizing VC7.1 compiler) Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. [It looks as if optimizing vs. non-optimizing is not something that you can detect by looking at the version number.] Could you please provide me with more version numbers and logo printouts ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 01 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From hyeshik at gmail.com Wed Dec 1 16:43:12 2004 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Wed Dec 1 16:43:15 2004 Subject: [Python-Dev] MS VC compiler versions In-Reply-To: <41ADDED2.30707@egenix.com> References: <41ADDED2.30707@egenix.com> Message-ID: <4f0b69dc0412010743352b1a8e@mail.gmail.com> On Wed, 01 Dec 2004 16:10:10 +0100, M.-A. Lemburg <mal@egenix.com> wrote: > Could you please provide me with more version numbers and logo > printouts ? > * MS Windows XP DDK (International version, optimizing VC 7.0): Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9176 for 80x86 Copyright (C) Microsoft Corporation 1984-2001. All rights reserved. * MS VS6 SP5 (International version, optimizing VC 6.0): Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86 Copyright (C) Microsoft Corp 1984-1998. All rights reserved. Hye-Shik From theller at python.net Wed Dec 1 19:14:51 2004 From: theller at python.net (Thomas Heller) Date: Wed Dec 1 19:14:03 2004 Subject: [Python-Dev] Trouble installing 2.4 In-Reply-To: <000401c4d708$ba149dc0$6402a8c0@arkdesktop> (Andrew Koenig's message of "Tue, 30 Nov 2004 13:16:34 -0500") References: <000401c4d708$ba149dc0$6402a8c0@arkdesktop> Message-ID: <mzwxrilg.fsf@python.net> "Andrew Koenig" <ark-mlist@att.net> writes: > Follow-up: When I install Python as Administrator, all is well. In that > case (but not when installing it as me), it asks whether I want to install > it for all users or for myself only. I then install pywin32 and it works. > > So it may be that a caveat is in order to people who do not install 2.4 as > Administrator. As Martin guessed, a distutils bug, triggered with non-admin Python installs, and when the add-on package uses the pre-install-script or post-install-script option. Please submit a report to SF. Thanks, Thomas From trentm at ActiveState.com Wed Dec 1 19:54:31 2004 From: trentm at ActiveState.com (Trent Mick) Date: Wed Dec 1 19:55:50 2004 Subject: [Python-Dev] MS VC compiler versions In-Reply-To: <41ADDED2.30707@egenix.com> References: <41ADDED2.30707@egenix.com> Message-ID: <41AE1367.5000805@activestate.com> M.-A. Lemburg wrote: > Preparing for the distutils patch to allow building extensions > using the .NET SDK compilers, I am compiling a list of version > numbers and MS compiler logo outputs in order to use these to > identify the correct compiler to use for the extensions. > > These are the compilers I have found so far: > > * MS VC6 (German version; optimizing VC6 compiler): > > Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, > fuer x86 > Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten. * MS VC6 (US version; optimizing VC6 compiler): Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86 Copyright (C) Microsoft Corp 1984-1998. All rights reserved. Trent -- Trent Mick trentm@activestate.com From aahz at pythoncraft.com Wed Dec 1 20:25:57 2004 From: aahz at pythoncraft.com (Aahz) Date: Wed Dec 1 20:25:59 2004 Subject: [Python-Dev] Roster Deadline In-Reply-To: <41AD5822.2000501@ieee.org> References: <41AD5822.2000501@ieee.org> Message-ID: <20041201192557.GA24980@panix.com> On Tue, Nov 30, 2004, Tim Hochberg wrote: > > Hi Larry, > > FYI: I asked EB about the roster deadline and she says that she doesn't > know when it is either. Checking on the Lei Out web page didn't help > much either. > > So, you are no wiser now than at the start of this message. We're even less wise now given that you probably didn't intend this for python-dev. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ WiFi is the SCSI of the 21st Century -- there are fundamental technical reasons for sacrificing a goat. (with no apologies to John Woods) From steven.bethard at gmail.com Wed Dec 1 22:03:50 2004 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed Dec 1 22:03:54 2004 Subject: [Python-Dev] adding key argument to min and max Message-ID: <d11dcfba0412011303463121d3@mail.gmail.com> This is my first post to Python dev, so I figured I should introduce myself. My name's Steven Bethard and I'm a computer science Ph.D. student at the University of Colorado at Boulder working primarily in the areas of natural language processing and machine learning. During my undergrad at the University of Arizona, I worked as a teaching assistant teaching Java for 2 1/2 years, though now that I'm at CU Boulder I pretty much only work in Python. I started getting active on the Python list about 6 months ago, and I've been watching python-dev for the last few months. On to the real question... I posted a few notes about this on the python-list and didn't hear much of a response, so I thought that maybe python-dev is the more appropriate place (since it involves a change to some of the builtin functions). For Python 2.5, I'd like to add a keyword argument 'key' to min and max like we have now for list.sort and sorted. I've needed this a couple of times now, specifically when I have something like a dict of word counts, and I want the most frequent word, I'd like to do something like: >>> d = dict(aaa=3000, bbb=2000, ccc=1000) >>> max(d, key=d.__getitem__) 'aaa' I've implemented a patch that provides this functionality, but there are a few concerns about how it works. Here's some examples of what it does now: >>> d = dict(aaa=3000, bbb=2000, ccc=1000) >>> max(d) 'ccc' >>> max(d, key=d.__getitem__) 'aaa' >>> max(d, d.__getitem__) {'aaa': 3000, 'bbb': 2000, 'ccc': 1000} >>> max(('aaa', 3000), ('bbb', 2000), ('ccc', 1000)) ('ccc', 1000) >>> max(('aaa', 3000), ('bbb', 2000), ('ccc', 1000), key=operator.itemgetter(1)) ('aaa', 3000) >>> max(('aaa', 3000), ('bbb', 2000), ('ccc', 1000), operator.itemgetter(1)) ('ccc', 1000) Note the difference between the 2nd and 3rd use of max in each example. For backwards compatibility reasons, the 'key' argument cannot be specified as a positional argument or it will look like max is being used in the max(a, b, c, ...) form. This means that a 'key' argument can *only* be specified as a keyword parameter, thus giving us the asymmetry we see in these examples. My real question then is, is this asymmetry a problem? Is it okay to have a parameter that is *only* accessable as a keyword parameter? Thanks, Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From bac at OCF.Berkeley.EDU Wed Dec 1 22:42:02 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Dec 1 22:42:20 2004 Subject: [Python-Dev] TRUNK UNFROZEN; release24-maint branch has been cut In-Reply-To: <200412010023.17728.anthony@interlink.com.au> References: <200412010023.17728.anthony@interlink.com.au> Message-ID: <41AE3AAA.7010402@ocf.berkeley.edu> Anthony Baxter wrote: > I've cut the release24-maint branch, and updated the Include/patchlevel.h > on trunk and branch (trunk is now 2.5a0, branch is 2.4+) > > The trunk and the branch are now both unfrozen and suitable for checkins. > The feature freeze on the trunk is lifted. Remember - if you're checking > bugfixes into the trunk, either backport them to the branch, or else mark > the commit message with 'bugfix candidate' or 'backport candidate' or the > like. > > Next up will be a 2.3.5 release. I'm going to be travelling for a large chunk > of December (at very short notice) so it's likely that this will happen at the > start of January. OK, I will send out an email to python-list and python-announce mentioning this to the community and that if they have fixes they need to go into 2.3.5 they need to get it in ASAP so there is enough time to consider them (with no guarantee they will get in) at the end of the week if no one objects. -Brett From pje at telecommunity.com Wed Dec 1 23:10:07 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Dec 1 23:08:35 2004 Subject: [Python-Dev] adding key argument to min and max In-Reply-To: <d11dcfba0412011303463121d3@mail.gmail.com> Message-ID: <5.1.1.6.0.20041201170900.02532c40@mail.telecommunity.com> At 02:03 PM 12/1/04 -0700, Steven Bethard wrote: >Is it okay to >have a parameter that is *only* accessable as a keyword parameter? Yes. From python at rcn.com Thu Dec 2 00:23:29 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Dec 2 00:26:09 2004 Subject: [Python-Dev] adding key argument to min and max In-Reply-To: <d11dcfba0412011303463121d3@mail.gmail.com> Message-ID: <001e01c4d7fc$c49efa20$e841fea9@oemcomputer> [Steven Bethard] > For Python 2.5, I'd like to add a keyword argument 'key' to min and > max like we have now for list.sort and sorted. . . . > This means that a 'key' > argument can *only* be specified as a keyword parameter, thus giving > us the asymmetry we see in these examples. FWIW, in Py2.5 I plan on adding a key= argument to heapq.nsmallest() and heapq.nlargest(). There is no "assymmetry" issue with those functions, so it can be implemented cleanly. And, since min/max are essentially the same nsmallest/nlargest with n==1, your use case is covered and there is no need to mess with the min/max builtins. > I've needed this a couple of times now, specifically when > I have something like a dict of word counts, and I want the most frequent word For Py2.4, you can cover your use cases readily adding the recipe for mostcommon() to a module of favorite utilities: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/347615 Alternatively, the recipe for a bag class is a more flexible and still reasonably efficient: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/259174 Raymond Hettinger From steven.bethard at gmail.com Thu Dec 2 02:02:33 2004 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu Dec 2 02:02:36 2004 Subject: [Python-Dev] adding key argument to min and max In-Reply-To: <001e01c4d7fc$c49efa20$e841fea9@oemcomputer> References: <d11dcfba0412011303463121d3@mail.gmail.com> <001e01c4d7fc$c49efa20$e841fea9@oemcomputer> Message-ID: <d11dcfba041201170275239b1b@mail.gmail.com> Raymond Hettinger <python@rcn.com> wrote: > [Steven Bethard] > > For Python 2.5, I'd like to add a keyword argument 'key' to min and > > max like we have now for list.sort and sorted. > . . . > > This means that a 'key' > > argument can *only* be specified as a keyword parameter, thus giving > > us the asymmetry we see in these examples. > > FWIW, in Py2.5 I plan on adding a key= argument to heapq.nsmallest() and > heapq.nlargest(). There is no "assymmetry" issue with those functions, > so it can be implemented cleanly. And, since min/max are essentially > the same nsmallest/nlargest with n==1, your use case is covered and > there is no need to mess with the min/max builtins. I don't want to put words into your mouth, so is this a vote against a key= argument for min and max? If nsmallest/nlargest get key= arguments, this would definitely cover the same cases. If a key= argument gets vetoed for min and max, I'd at least like to add a bit of documentation pointing users of min/max to nsmallest/nlargest if they want a key= argument... Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From python at rcn.com Thu Dec 2 02:32:10 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Dec 2 02:34:51 2004 Subject: [Python-Dev] adding key argument to min and max In-Reply-To: <d11dcfba041201170275239b1b@mail.gmail.com> Message-ID: <002301c4d80e$beb08e00$e841fea9@oemcomputer> > I don't want to put words into your mouth, so is this a vote against a > key= argument for min and max? Right. I don't think there is any need. > If nsmallest/nlargest get key= arguments, this would definitely cover > the same cases. Right. > If a key= argument gets vetoed for min and max, I'd > at least like to add a bit of documentation pointing users of min/max > to nsmallest/nlargest if they want a key= argument... Sounds reasonable. Raymond P.S. In case you're interested, here is the patch: Index: heapq.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/heapq.py,v retrieving revision 1.27 diff -u -r1.27 heapq.py --- heapq.py 29 Nov 2004 05:54:47 -0000 1.27 +++ heapq.py 2 Dec 2004 01:32:44 -0000 @@ -307,6 +307,31 @@ except ImportError: pass +# Extend the implementations of nsmallest and nlargest to use a key= argument +_nsmallest = nsmallest +def nsmallest(n, iterable, key=None): + """Find the n smallest elements in a dataset. + + Equivalent to: sorted(iterable, key=key)[:n] + """ + if key is None: + return _nsmallest(n, iterable) + it = ((key(r), i, r) for i, r in enumerate(iterable)) # decorate + result = _nsmallest(n, it) + return [r for k, i, r in result] # undecorate + +_nlargest = nlargest +def nlargest(n, iterable, key=None): + """Find the n largest elements in a dataset. + + Equivalent to: sorted(iterable, key=key, reverse=True)[:n] + """ + if key is None: + return _nlargest(n, iterable) + it = ((key(r), i, r) for i, r in enumerate(iterable)) # decorate + result = _nlargest(n, it) + return [r for k, i, r in result] # undecorate + if __name__ == "__main__": # Simple sanity test heap = [] Index: test/test_heapq.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_heapq.py,v retrieving revision 1.16 diff -u -r1.16 test_heapq.py --- test/test_heapq.py 29 Nov 2004 05:54:48 -0000 1.16 +++ test/test_heapq.py 2 Dec 2004 01:32:44 -0000 @@ -105,13 +105,19 @@ def test_nsmallest(self): data = [random.randrange(2000) for i in range(1000)] + f = lambda x: x * 547 % 2000 for n in (0, 1, 2, 10, 100, 400, 999, 1000, 1100): self.assertEqual(nsmallest(n, data), sorted(data)[:n]) + self.assertEqual(nsmallest(n, data, key=f), + sorted(data, key=f)[:n]) def test_largest(self): data = [random.randrange(2000) for i in range(1000)] + f = lambda x: x * 547 % 2000 for n in (0, 1, 2, 10, 100, 400, 999, 1000, 1100): self.assertEqual(nlargest(n, data), sorted(data, reverse=True)[:n]) + self.assertEqual(nlargest(n, data, key=f), + sorted(data, key=f, reverse=True)[:n]) #======================================================================= ======= From gvanrossum at gmail.com Thu Dec 2 02:58:48 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Dec 2 02:58:51 2004 Subject: [Python-Dev] adding key argument to min and max In-Reply-To: <002301c4d80e$beb08e00$e841fea9@oemcomputer> References: <d11dcfba041201170275239b1b@mail.gmail.com> <002301c4d80e$beb08e00$e841fea9@oemcomputer> Message-ID: <ca471dc204120117582c69d6ac@mail.gmail.com> > > I don't want to put words into your mouth, so is this a vote against a > > key= argument for min and max? > > Right. I don't think there is any need. Hm, min and max are probably needed 2-3 orders of magnitude more frequently than nsmallest/nlargest. So I think it's reasonable to add the key= argument to min and max as well. (We didn't leave it out of sorted() because you can already do it with list.sort().) > def test_largest(self): shouldn't that be test_nlargest? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Dec 2 03:04:57 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Dec 2 03:07:36 2004 Subject: [Python-Dev] adding key argument to min and max In-Reply-To: <d11dcfba0412011303463121d3@mail.gmail.com> Message-ID: <002e01c4d813$529cc300$e841fea9@oemcomputer> > -----Original Message----- > From: python-dev-bounces+python=rcn.com@python.org [mailto:python-dev- > bounces+python=rcn.com@python.org] On Behalf Of Steven Bethard > Sent: Wednesday, December 01, 2004 4:04 PM > To: python-dev@python.org > Subject: [Python-Dev] adding key argument to min and max > > This is my first post to Python dev, so I figured I should introduce > myself. > > My name's Steven Bethard and I'm a computer science Ph.D. student at > the University of Colorado at Boulder working primarily in the areas > of natural language processing and machine learning. During my > undergrad at the University of Arizona, I worked as a teaching > assistant teaching Java for 2 1/2 years, though now that I'm at CU > Boulder I pretty much only work in Python. I started getting active > on the Python list about 6 months ago, and I've been watching > python-dev for the last few months. > > For Python 2.5, I'd like to add a keyword argument 'key' to min and > max like we have now for list.sort and sorted. . . . > I've implemented a patch that provides this functionality, but there > are a few concerns about how it works. Guido says yes. So, load the patch to SF and assign to me for review, testing, and documentation. Raymond From kbk at shore.net Thu Dec 2 04:15:53 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Dec 2 04:15:57 2004 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <873byp1jbq.fsf@hydra.localdomain> Patch / Bug Summary ___________________ Patches : 258 open ( +4) / 2701 closed ( +1) / 2959 total ( +5) Bugs : 812 open (+28) / 4642 closed (+13) / 5454 total (+41) RFE : 160 open ( +4) / 136 closed ( +1) / 296 total ( +5) New / Reopened Patches ______________________ #1074261 gzip dies on gz files with many appended headers (2004-11-27) http://python.org/sf/1074381 opened by Mark Eichin Flush stdout/stderr if closed (fix for bug 1074011) (2004-11-29) http://python.org/sf/1075147 opened by Ben Hutchings gcc compiler on Windows (2004-11-30) http://python.org/sf/1075887 opened by Michiel de Hoon AUTH PLAIN in smtplib (2004-11-30) http://python.org/sf/1075928 opened by James Lan readline does not need termcap (2004-12-01) http://python.org/sf/1076826 opened by Michal Čihař Patches Closed ______________ bug#1021756: more informative error message (2004-11-23) http://python.org/sf/1071739 closed by effbot New / Reopened Bugs ___________________ Bugs in _csv module - lineterminator (2004-11-24) http://python.org/sf/1072404 opened by Chris Withers mkarg undocumented (2004-11-24) http://python.org/sf/1072410 opened by Gunter Ohrner email as_string() omits trailing newline (2004-11-24) CLOSED http://python.org/sf/1072623 opened by Tessa Lau dyld: ./python.exe multiple definitions of symbol _BC (2004-11-24) http://python.org/sf/1072642 opened by Marius thisid not intialized in pindent.py script (2004-11-24) http://python.org/sf/1072853 opened by Niraj Bajpai ^Z doesn't exit interpreter - 2.4c1 & Win2K (2004-11-26) http://python.org/sf/1073736 opened by Kent Johnson subprocess.py doc bug in 2.4c1 (2004-11-26) CLOSED http://python.org/sf/1073790 opened by Dan Christensen 2 XML parsing errors (2004-11-26) CLOSED http://python.org/sf/1073864 opened by Peer Janssen write failure ignored in Py_Finalize() (2004-11-27) http://python.org/sf/1074011 opened by Matthias Klose current directory in sys.path handles symlinks badly (2004-11-26) http://python.org/sf/1074015 opened by Eric M. Hopper xml.dom.minidom produces errors with certain unicode chars (2004-11-27) http://python.org/sf/1074200 opened by Peer Janssen gzip dies on gz files with many appended headers (2004-11-27) http://python.org/sf/1074261 opened by Mark Eichin input from numeric pad always dropped when numlock off (2004-11-27) http://python.org/sf/1074333 opened by Rick Graves FeedParser problem on end boundaries w/o newline (2004-11-24) http://python.org/sf/1072623 reopened by tlau Irregular behavior of datetime.__str__() (2004-11-27) http://python.org/sf/1074462 opened by Wai Yip Tung Errors and omissions in logging module documentation (2004-11-28) http://python.org/sf/1074693 opened by Joachim Boomberschloss Windows 2.4c1 installer default location issues (2004-11-28) http://python.org/sf/1074873 opened by dmerrill exceeding obscure weakproxy bug (2004-11-29) http://python.org/sf/1075356 opened by Michael Hudson urllib2.HTTPBasicAuthHandler problem with [HOST]:[PORT] (2004-11-29) http://python.org/sf/1075427 opened by O-Zone PyGILState_Ensure() deadlocks (ver 2.4c1) (2004-11-29) http://python.org/sf/1075703 opened by Andi Vajda Build Bug on Solaris. (2004-11-30) CLOSED http://python.org/sf/1075934 opened by Jeremy Whiting Memory fault pyexpat.so on SGI (2004-11-30) http://python.org/sf/1075984 opened by Maik Hertha Memory fault pyexpat.so on SGI (2004-11-30) http://python.org/sf/1075990 opened by Maik Hertha HTMLParser can't handle page with javascript (2004-11-30) http://python.org/sf/1076070 opened by Jeremy Hylton distutils.core.setup() with unicode arguments broken (2004-11-30) http://python.org/sf/1076233 opened by Walter D?rwald Whats New for 2.4 "SafeTemplate" patch. (2004-11-30) CLOSED http://python.org/sf/1076365 opened by Sean Reifschneider test_shutil fails on x86-64 // Suse 9.1 (2004-11-30) http://python.org/sf/1076467 opened by Ross G Baker Jr Another message that croaks email.FeedParser (2004-11-30) http://python.org/sf/1076485 opened by Skip Montanaro Sate/Save typo in Mac/scripts/BuildApplication.py (2004-11-30) http://python.org/sf/1076490 opened by Neil Mayhew BuildApplication includes many unneeded modules (2004-11-30) http://python.org/sf/1076492 opened by Neil Mayhew python24.msi install error (2004-12-01) http://python.org/sf/1076500 opened by guan zi jing shutil.move clobbers read-only files. (2004-12-01) http://python.org/sf/1076515 opened by Jeremy Fincher test test_codecs failed (2004-12-01) http://python.org/sf/1076790 opened by Michal Čihař test test_re produced unexpected output (2004-12-01) http://python.org/sf/1076791 opened by Michal Čihař test test_unicodedata failed (2004-12-01) http://python.org/sf/1076795 opened by Michal Čihař test test_shutil failed (2004-12-01) CLOSED http://python.org/sf/1076811 opened by Michal Čihař test_shelve failures (2004-12-01) http://python.org/sf/1076819 opened by Michal Čihař Bad IDLE shortcut by 2.4 installer on XP (2004-12-01) http://python.org/sf/1076861 opened by Jean M. Brouwers Tutorial corrections (2004-12-01) http://python.org/sf/1076955 opened by Fred L. Drake, Jr. Incorrect behaviour of StreamReader.readline leads to crash (2004-12-01) http://python.org/sf/1076985 opened by Sebastian Hartte Error building _bsddb extension (2004-12-01) http://python.org/sf/1077040 opened by Oleg Broytmann Problem testing python 2.4 (2004-12-01) http://python.org/sf/1077103 opened by Pierre Negative numbers to os.read() cause segfault (2004-12-01) http://python.org/sf/1077106 opened by Jp Calderone Bugs Closed ___________ coercing decimal to int doesn't work between -1 and 1 (2004-11-23) http://python.org/sf/1071588 closed by rhettinger OS X (Panther) Framework Install causes version mismatch (2004-11-18) http://python.org/sf/1069198 closed by bcannon urllib2 authentication redirection error(?) (2004-11-22) http://python.org/sf/1071147 closed by facundobatista shelve update fails on "large" entry (2002-02-27) http://python.org/sf/523425 closed by facundobatista new int overflow handling needs docs (2001-08-23) http://python.org/sf/454446 closed by facundobatista subprocess.py doc bug in 2.4c1 (2004-11-26) http://python.org/sf/1073790 closed by astrand 2 XML parsing errors (2004-11-26) http://python.org/sf/1073864 closed by peerjanssen gzip module cannot write large files (2003-02-08) http://python.org/sf/683061 closed by jlgijsbers FeedParser problem on end boundaries w/o newline (2004-11-24) http://python.org/sf/1072623 closed by bwarsaw FeedParser problem on end boundaries w/o newline (2004-11-24) http://python.org/sf/1072623 closed by bwarsaw Build Bug on Solaris. (2004-11-30) http://python.org/sf/1075934 closed by whitingjr Whats New for 2.4 "SafeTemplate" patch. (2004-11-30) http://python.org/sf/1076365 closed by akuchling 'Demo/embed/' , It crash (2003-12-04) http://python.org/sf/853862 closed by guanzijing test test_shutil failed (2004-12-01) http://python.org/sf/1076811 closed by nijel New / Reopened RFE __________________ Extension to optparse: options with facultative value (2004-11-25) http://python.org/sf/1073198 opened by pollastri Extension to optparse: options with facultative value (2004-11-25) CLOSED http://python.org/sf/1073305 opened by pollastri ignore element format character for PyArg_ParseTuple (2004-11-30) http://python.org/sf/1075902 opened by Sean Proctor Python Interpreter embedded in small memory system (2004-12-01) http://python.org/sf/1076478 opened by stoneabcwujx RFE Closed __________ Extension to optparse: options with facultative value (2004-11-25) http://python.org/sf/1073305 closed by nnorwitz From phd at phd.pp.ru Wed Dec 1 21:47:21 2004 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu Dec 2 11:46:39 2004 Subject: [Python-Dev] Python 2.4 README Message-ID: <20041201204721.GA6993@phd.pp.ru> Hello. Python 2.4 README still thinks it is 2.3. README lines 230-231: "A number of features are not supported in Python 2.3 anymore. Some support code is still present, but will be removed in Python 2.4." "Will be"? Isn't it 2.4 already? :) Line 322: "Tcl to support it. To compile Python2.3 with Tkinter, you will" Python2.4, I suppose? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From bjoti at home.se Wed Dec 1 23:19:22 2004 From: bjoti at home.se (Bjorn Tillenius) Date: Thu Dec 2 11:46:40 2004 Subject: [Python-Dev] Unicode in doctests Message-ID: <20041201221922.GA26263@home.se> There are some issues regarding the use of unicode in doctests. Consider the following three tests. >>> foo = u'f??' >>> foo u'f\xf6\xf6' >>> foo u'f\u00f6\u00f6' >>> foo u'f??' To me, those are identical. At least string comparison shows that u'f\xf6\xf6' == u'f\u00f6\u00f6' == u'f??'. Yet, only the first one of the tests passes, the other two fail. And that's if the tests are within a doc string, where I can specify the encoding used. If DocFileSuite is being used, there's no way of specify the encoding, thus all tests will fail. Is it supposed to be like this, or have I missed something? If I could specify the encoding for DocFileSuite to use, I would at least be partially happy. Regards, Bjorn From niemeyer at conectiva.com Thu Dec 2 17:31:06 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Dec 2 17:31:13 2004 Subject: [Python-Dev] SRE bug and notifications Message-ID: <20041202163106.GA16046@burma.localdomain> I'm really ashamed. The SRE bug #1072259, reported in 2004-11-24, and fixed a few minutes ago, got into 2.4 final. The only reason it wasn't fixed on time for 2.4 is because I wasn't aware about it. :-( Is there any way to get notified about certain kinds of bugs in SourceForge? Or, is there any way to follow all new bugs posted? Or even, what's the easiest way to avoid that problem by being notified of bugs as soon as possible? Thank you, and I'm sorry. -- Gustavo Niemeyer http://niemeyer.net From amk at amk.ca Thu Dec 2 17:46:07 2004 From: amk at amk.ca (A.M. Kuchling) Date: Thu Dec 2 17:46:12 2004 Subject: [Python-Dev] SRE bug and notifications In-Reply-To: <20041202163106.GA16046@burma.localdomain> References: <20041202163106.GA16046@burma.localdomain> Message-ID: <20041202164607.GA8027@rogue.amk.ca> On Thu, Dec 02, 2004 at 02:31:06PM -0200, Gustavo Niemeyer wrote: > in SourceForge? Or, is there any way to follow all new bugs > posted? Or even, what's the easiest way to avoid that problem > by being notified of bugs as soon as possible? Kurt's weekly bug summaries list all new bugs. For a sample, see http://mail.python.org/pipermail/python-list/2004-December/252968.html ; they get posted to python-dev, too. Perhaps the summaries should include an "unassigned bugs" list to nag us to look at bugs and assign them to the right person. --amk From niemeyer at conectiva.com Thu Dec 2 17:48:54 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Dec 2 17:49:00 2004 Subject: [Python-Dev] Re: SRE bug and notifications In-Reply-To: <20041202163106.GA16046@burma.localdomain> References: <20041202163106.GA16046@burma.localdomain> Message-ID: <20041202164854.GA16821@burma.localdomain> > Is there any way to get notified about certain kinds of bugs > in SourceForge? Or, is there any way to follow all new bugs > posted? Or even, what's the easiest way to avoid that problem > by being notified of bugs as soon as possible? After some googling I've found python-bugs-list. Mentioning it in http://www.python.org/community/lists.html might be a good idea. Additional suggestions still accepted. Thanks! -- Gustavo Niemeyer http://niemeyer.net From tim.peters at gmail.com Thu Dec 2 18:02:51 2004 From: tim.peters at gmail.com (Tim Peters) Date: Thu Dec 2 18:02:53 2004 Subject: [Python-Dev] SRE bug and notifications In-Reply-To: <20041202163106.GA16046@burma.localdomain> References: <20041202163106.GA16046@burma.localdomain> Message-ID: <1f7befae041202090237b3d159@mail.gmail.com> FYI, I just changed SF so that bugs with Category "Regular Expressions" are auto-assigned to Gustavo (they were being auto-assigned to Fredrik, which doesn't appear to do much good anymore). [Gustavo Niemeyer] > I'm really ashamed. The SRE bug #1072259, reported in > 2004-11-24, and fixed a few minutes ago, got into 2.4 final. The > only reason it wasn't fixed on time for 2.4 is because I wasn't > aware about it. :-( Unfortunately, that particular bug was in a wrong Category, so the new auto-assign wouldn't have helped here anyway. From python at rcn.com Fri Dec 3 00:17:25 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Dec 3 00:20:05 2004 Subject: [Python-Dev] [Python-checkins] python/dist/src/Doc/lib liblogging.tex, 1.33, 1.34 Message-ID: <006301c4d8c5$15972380$e841fea9@oemcomputer> Reminder: The head is now for Py2.5. Please also do a checkin for release24-maint. Raymond From noamr at myrealbox.com Fri Dec 3 01:55:28 2004 From: noamr at myrealbox.com (Noam Raphael) Date: Fri Dec 3 01:55:16 2004 Subject: [Python-Dev] An import hook which does nothing Message-ID: <41AFB980.3050705@myrealbox.com> Hello, I'm currently writing an import hook which will cache information on the local disk, instead of retrieving it every time from the slow NFS where I work. To make sure that I understand how import hooks work, and as a starting point, I wrote a dummy import hook, which is supposed to behave like the python's built-in import mechanism. I post it here, for two reasons: 1. So that developers who understand the import mechanism better than I do may review it and find things which I didn't do exactly right. 2. Because I think other people might find it helpful when writing new import hooks, as a starting point as well as for better understanding -- there's nothing like a working example to help you settle up on what does which where. (Although perhaps a few more comments, in addition to those which I wrote, might help...) (Note: I wrote the "DEBUG" parts in order to make sure that my importer works, because if it fails things might be done by the built-in importer and I won't notice. These parts can be removed, of course.) Do you think that it might be useful? Maybe something like that can go into the "examples" section of the imp module? Thanks, Noam Raphael -------------- next part -------------- A non-text attachment was scrubbed... Name: importer.py Type: text/x-python Size: 1910 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20041203/378063b9/importer.py From sxanth at ceid.upatras.gr Fri Dec 3 10:54:25 2004 From: sxanth at ceid.upatras.gr (Stelios Xanthakis) Date: Fri Dec 3 10:54:31 2004 Subject: [Python-Dev] PEP: __source__ proposal Message-ID: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> Hi all. Now that 2.4 is out and everything maybe it's about time to start discussing the "use the __source__ Luke" feature which IMO will really boost python into a new domain of exciting possibilities. I've prepared a pre-PEP which is not very good but it is a base. In short, the feature is good and it enables editing of python code at runtime instead of the runfile-exit-edit-run-exit-edit-run cycle. We have the following possibilities as to whether __source__ data is marshalled and the feature is always enabled. [1] Command line switch and not marshalled [2] Always on and not marshalled [3] Always on and marshalled There is also [4] which doesn't make much sense. If I was BDFL I'd go for [1] so whoever wants it can enable it and whoever doesn't can't complain, and they'll all leave me alone. Phillip J. Eby expressed some concerns that the modules that depend on __source__ will eventually take over and it will become a standard. Anyway, the PEP is attached. You can mail me with votes on the feature and if you want on your preferred option from 1,2,3. If I get votes I'll post the results later. If this is accepted I'll try to come up with a good patch vs 2.4 Thanks, St. -------------------ATTACHED PYTHON ENHANCEMENT PROPOSAL--- PEP: XXX Title: The __source__ attribute Version: $Revision: 1.10 $ Last-Modified: $Date: 2003/09/22 04:51:49 $ Author: Stelios Xanthakis Status: Draft Type: Standards Track Content-Type: text/plain Created: 19-Nov-2004 Python-Version: 2.4.1 Post-History: Abstract This PEP suggests the implementation of __source__ attribute for functions and classes. The attribute is a read-only string which is generated by the parser and is a copy of the original source code of the function/class (including comments, indentation and whitespace). Motivation It is generally a tempting idea to use python as an interface to a program. The developers can implement all the functionality and instead of designing a user interface, provide a python interpreter to their users. Take for example one of the existing web browsers: they have everything that would be needed to write a script which downloads pages automatically or premutates the letters of web pages before they are displayed, but it is not possible for the user to do these things because the interface of these applications is static. A much more powerful approach would be an interface which is dynamically constructed by the user to meet the user's needs. The most common development cycle of python programs is: write .py file - execute .py file - exit - enhance .py file - execute .py file - etc. With the implementation of the __source__ attribute though the development/modification of python code can happen at run-time. Functions and classes can be defined, modified or enhanced while the python shell is running and all the changes can be saved by saving the __source__ attribute of globals before termination. Moreover, in such a system it is possible to modify the "code modification routines" and eventually we have a self-modifying interface. Using a program also means improving its usability. The current solution of using 'inspect' to get the source code of functions is not adequate because it doesn't work for code defined with "exec" and it doesn't have the source of functions/classes defined in the interactive mode. Generally, a "file" is something too abstract. What is more real is the data received by the python parser and that is what is stored in __source__. Specification The __source__ attribute is a read-only attribute of functions and classes. Its type is string or None. In the case of None it means that the source was not available. The indentation of the code block is the original identation obeying nested definitions. For example: >>> class A: ... def foo (self): ... print """Santa-Clauss ... is coming to town""" >>> def spam (): ... def closure (): ... pass ... return closure >>> print A.foo.__source__ def foo (self): print """Santa-Clauss is coming to town""" >>> print spam().__source__ def closure (): pass The attribute is not marshaled and therefore not stored in ".pyc" files. As a consequence, functions and classes of imported modules have __source__==None. We propose that the generation of __source__ will be controlled by a command line option. In the case this feature is not activated by the command line option, the attribute is absent. Rationale Generally, "import" refers to modules that either have a file in a standard location or they are distributed in ".pyc" form only. Therefore in the case of modules, getting the source with "inspect" is adequate. Moreover, it does not make sense saving __source__ in ".pyc" because the point would be to save modifications in the original ".py" file (if available). On the issue of the command-line option controlling the generation of __source__, please refer to the section about the overhead of this feature. The rationale is that those applications that do not wish to use this feature can avoid it (cgi scripts in python benchmarked against another language). Overhead The python's parser is not exactly well-suited for such a feature. Execution of python code goes through the stages of lexical analysis, tokenization, generation of AST and execution of bytecode. In order to implement __source__, the tokenizer has to be modified to store the lines of the current translation unit. Those lines are then attached the root node of the AST. While the AST is compiled we have to keep a reference of the current node in order to be able to find the next node after the node for which we wish to generate __source__, get the first and the last line of our block and then refer to the root node to extract these lines and make a string. All these actions add a minor overhead to some heavily optimized parts of python. However, once compilation to bytecode is done, this feature no longer affects the performance of the execution of the bytecode. There is also the issue of the memory spent to store __source__. In our opinion, this is worth the tradeoff for those who are willing to take advantage of it. Implementation There is a sample implementation at [2] which consists of a patch against python 2.3.4. The patch has to be improved to avoid generating __source__ for the case we are importing modules for the first time (not from .pyc). In the sample implementation there is also included a sample shell that takes advantage of __source__ and demonstrates some aspects that motivated us towards patching python and submitting this PEP. References [1] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html [2] Sample implementation http://students.ceid.upatras.gr/~sxanth/ISYSTEM/python-PIESS.tar.gz Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From hpk at trillke.net Fri Dec 3 13:36:16 2004 From: hpk at trillke.net (holger krekel) Date: Fri Dec 3 13:36:20 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> Message-ID: <20041203123616.GB4708@solar.trillke.net> Hi Stelios, [Stelios Xanthakis Fri, Dec 03, 2004 at 11:54:25AM +0200] > Abstract > > This PEP suggests the implementation of __source__ attribute for > functions and classes. The attribute is a read-only string which > is generated by the parser and is a copy of the original source > code of the function/class (including comments, indentation and > whitespace). I've had similar ideas in the past as we are doing dynamic code generation in PyPy, as well as in other projects. After some discussion with Armin i think there is another possibility for storing "source" or any other such meta information with code/module objects: make __file__ and co_filename instances of a subclass of 'str' providing an extra attribute. For a simple example, they could have a 'source' attribute, which could be tried first by appropriate inspect functions and traceback related functionality. We are about to test out this approach with the py lib (http://codespeak.net/py) and want to have it work for for Python 2.2, 2.3. and 2.4. I suspect there may be some issues lurking (also in your proposed PEP) especially with respect to encodings. Also we have some use cases where we want to retrieve source code from non-local locations and want to integrate this seemlessly with the introspection facilities of Python which obviously is an important part of the equation. I can report back if there is interest. cheers, holger From fredrik at pythonware.com Fri Dec 3 14:38:04 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri Dec 3 14:38:13 2004 Subject: [Python-Dev] Re: Unicode in doctests References: <20041201221922.GA26263@home.se> Message-ID: <copq7t$4d4$1@sea.gmane.org> Bjorn Tillenius wrote: > There are some issues regarding the use of unicode in doctests. Consider > the following three tests. > > >>> foo = u'föö' > >>> foo > u'f\xf6\xf6' > > >>> foo > u'f\u00f6\u00f6' > > >>> foo > u'föö' > > To me, those are identical. really? if a function is expected to print "öl", is it alright to print "\u00f6l" instead? wouldn't you complain if your newspaper used Unicode escapes in headlines instead of Swedish characters? > Is it supposed to be like this, or have I missed something? If I could > specify the encoding for DocFileSuite to use, I would at least be > partially happy. repr() always generates the same output, no matter what encoding you use. just use repr, and you're done. </F> From mal at egenix.com Fri Dec 3 15:59:43 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Dec 3 15:59:48 2004 Subject: [Python-Dev] Removing --with-wctype-functions support Message-ID: <41B07F5F.9040207@egenix.com> I would like to remove the support for using libc wctype functions (e.g. towupper(), towlower(), etc.) from the code base. The reason is that compiling Python using this switch not only breaks the test suite, it also causes the functions .lower() and .upper() to become locale aware and creates wrong results as a result (which are hard to track down if you don't know whether Python was compiled with the wctype switch or not). The argument for having this switch at the time was to reduce the interpreter size. This was true for Python 1.6 since the Unicode type database was conditionally removed. Starting with Python 2.0 a different approach was taken which resulted in having to keep the type database in one piece - even with the switch enabled. As a result, performance became the only argument. However, because the libc functions are locale aware, the added overhead causes the libc functions not to perform better than the builtin Python versions. In the end, we loose complexity and reduce the Python configuration space by one dimension. Here's the bug that revealed the problem: https://sourceforge.net/tracker/index.php?func=detail&aid=1076790&group_id=5470&atid=105470 Question is: Should this be done in 2.4.1 or will it have to wait until 2.5 ? We might want to remove the support for 2.4.1 and simply ignore the compile time switch (or print a warning). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 03 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Dec 3 16:08:54 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Dec 3 16:08:58 2004 Subject: [Python-Dev] MS VC compiler versions In-Reply-To: <41AE1367.5000805@activestate.com> References: <41ADDED2.30707@egenix.com> <41AE1367.5000805@activestate.com> Message-ID: <41B08186.5070600@egenix.com> Here's an update of the list I currently have: * MS VS6 SP5 (International version, optimizing VC6 compiler with SP5): Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86 Copyright (C) Microsoft Corp 1984-1998. All rights reserved. * MS VC6 (German version; optimizing VC6 compiler with SP6): Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, fuer x86 Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten. * MS VC6 (US version; optimizing VC6 compiler with SP6): Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86 Copyright (C) Microsoft Corp 1984-1998. All rights reserved. * MS .NET SDK 1.1 Compiler (German version; non-optimizing VC7.1 compiler): Microsoft (R) 32-Bit C/C++-Standardcompiler Version 13.10.3077 f?r 80x86 Copyright (C) Microsoft Corporation 1984-2002. Alle Rechte vorbehalten. * MS VC7.1 (aka .NET 2003, German version; optimizing VC7.1 compiler) Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. * MS Windows XP DDK (International version, optimizing VC 7.0): Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9176 for 80x86 Copyright (C) Microsoft Corporation 1984-2001. All rights reserved. Does someone have the details for the MS Toolkit compiler ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 03 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From Scott.Daniels at Acm.Org Fri Dec 3 17:07:25 2004 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri Dec 3 17:06:06 2004 Subject: [Python-Dev] Re: MS VC compiler versions In-Reply-To: <41B08186.5070600@egenix.com> References: <41ADDED2.30707@egenix.com> <41AE1367.5000805@activestate.com> <41B08186.5070600@egenix.com> Message-ID: <coq2t8$ak$1@sea.gmane.org> M.-A. Lemburg wrote: > Here's an update of the list I currently have: > > * MS VS6 SP5 (International version, optimizing VC6 compiler with SP5): > > Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86 > Copyright (C) Microsoft Corp 1984-1998. All rights reserved. > > * MS VC6 (German version; optimizing VC6 compiler with SP6): > > Optimierender Microsoft (R) 32-Bit C/C++-Compiler, Version 12.00.8804, > fuer x86 > Copyright (C) Microsoft Corp 1984-1998. Alle Rechte vorbehalten. > > * MS VC6 (US version; optimizing VC6 compiler with SP6): > > Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86 > Copyright (C) Microsoft Corp 1984-1998. All rights reserved. > > * MS .NET SDK 1.1 Compiler (German version; non-optimizing VC7.1 compiler): > > Microsoft (R) 32-Bit C/C++-Standardcompiler Version 13.10.3077 f?r 80x86 > Copyright (C) Microsoft Corporation 1984-2002. Alle Rechte vorbehalten. > > * MS VC7.1 (aka .NET 2003, German version; optimizing VC7.1 compiler) > > Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 > Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. > > * MS Windows XP DDK (International version, optimizing VC 7.0): > > Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.00.9176 for 80x86 > Copyright (C) Microsoft Corporation 1984-2001. All rights reserved. > > > Does someone have the details for the MS Toolkit compiler ? > > Thanks, \Program Files\Microsoft Visual C++ Toolkit 2003\bin\cl.exe: Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. -- -- Scott David Daniels Scott.Daniels@Acm.Org From bjoti at yahoo.se Fri Dec 3 18:02:52 2004 From: bjoti at yahoo.se (Bjorn Tillenius) Date: Fri Dec 3 18:20:43 2004 Subject: [Python-Dev] Re: Unicode in doctests References: <20041201221922.GA26263@home.se> <copq7t$4d4$1@sea.gmane.org> Message-ID: <loom.20041203T173955-552@post.gmane.org> Fredrik Lundh <fredrik <at> pythonware.com> writes: > Bjorn Tillenius wrote: > > There are some issues regarding the use of unicode in doctests. Consider > > the following three tests. > > > > >>> foo = u'f??' > > >>> foo > > u'f\xf6\xf6' > > > > >>> foo > > u'f\u00f6\u00f6' > > > > >>> foo > > u'f??' > > > > To me, those are identical. > > really? if a function is expected to print "?l", is it alright to print > "\u00f6l" instead? wouldn't you complain if your newspaper used > Unicode escapes in headlines instead of Swedish characters? No, I wouldn't like the newspaper to use Unicode escapes. For the same reason, I don't want my documentation to contain Unicode escapes. That's why I would like the latter test to pass. But I understand, it tries to match the output of repr(foo), I guess I can live with that. I can always do: >>> foo == u'f??' True On the other hand, since there already are some flags to modify the matching algorithm, one could argue for adding another flag... or at least provide the possibility for the user to alter the matching himself. Although it's not that important for me. > > Is it supposed to be like this, or have I missed something? If I could > > specify the encoding for DocFileSuite to use, I would at least be > > partially happy. > > repr() always generates the same output, no matter what encoding > you use. just use repr, and you're done. What is important for me, though, is to be able to specify an encoding to DocFileSuite. As you said, one doesn't want to read Unicode escapes. At the moment none of the tests I've given as example will pass in a DocFileSuite (given that the text file is encoded using UTF-8). I do find it a bit strange that I can't just copy a doctest within a docstring to a text file. I have to Unicode escape all non-ASCII characters, which produces ugly documentation. Regards, Bjorn From kbk at shore.net Fri Dec 3 18:40:42 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Fri Dec 3 18:42:14 2004 Subject: [Python-Dev] SRE bug and notifications In-Reply-To: <20041202164607.GA8027@rogue.amk.ca> (A. M. Kuchling's message of "Thu, 2 Dec 2004 11:46:07 -0500") References: <20041202163106.GA16046@burma.localdomain> <20041202164607.GA8027@rogue.amk.ca> Message-ID: <87653jxoth.fsf@hydra.localdomain> "A.M. Kuchling" <amk@amk.ca> writes: > Perhaps the summaries should include an "unassigned bugs" list to nag > us to look at bugs and assign them to the right person. The bug report is derived from the bug and patch email lists, so that information isn't available without scraping it off SF. However, the SF trackers can be set to query for Unassigned / Open. It looks like about 2/3 of the open bugs and 3/4 of the open patches are unassigned. I often unassign IDLE bugs that I'm not planning to fix right now, hoping that may encourage someone to step forward by removing the illusion that something is happening. Conversely, if I make progress on an unassigned bug/patch, I'll assign it to myself as an indication that it's being worked on, avoiding duplication of effort. I track IDLE bugs via the IDLE category. That's very useful, maybe we need more categories? -- KBK From sxanth at ceid.upatras.gr Fri Dec 3 22:59:30 2004 From: sxanth at ceid.upatras.gr (Stelios Xanthakis) Date: Fri Dec 3 22:59:40 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <20041203123616.GB4708@solar.trillke.net> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> <20041203123616.GB4708@solar.trillke.net> Message-ID: <Pine.GSO.4.61.0412031554001.5550@zenon.ceid.upatras.gr> On Fri, 3 Dec 2004, holger krekel wrote: > ... > there is another possibility for storing "source" or any other such meta > information with code/module objects: make __file__ and co_filename > instances of a subclass of 'str' providing an extra attribute. > For a simple example, they could have a 'source' attribute, which > could be tried first by appropriate inspect functions and traceback > related functionality. Attaching such info on 'code objects' is indeed a more general case. But, OTOH, AFAIK, a class is not a code object. At least by what I was able to figure out from python sources. It seems reasonable to make 'source' a dynamic object which will get its info from file/line if available. Now the thing is that if we had __source__ from the start, 'inspect' would have been much different. So the fact that we have some functionality with inspect does not mean that it's good enough. Probably inspect will be rewritten/improved if __source__ is implemented. > We are about to test out this approach with the py lib > (http://codespeak.net/py) and want to have it work for > for Python 2.2, 2.3. and 2.4. Do you plan hacking python ? It appears that tok_nextc() is the best place to catch all the source passed to the interpreter. A patch would be interesting. Stelios From martin at v.loewis.de Sat Dec 4 00:26:33 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 4 00:26:32 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> Message-ID: <41B0F629.4040100@v.loewis.de> Stelios Xanthakis wrote: > Now that 2.4 is out and everything maybe it's > about time to start discussing the "use the > __source__ Luke" feature which IMO will really > boost python into a new domain of exciting > possibilities. I'm opposed to this idea. It creates overhead in the size of .pyc files, for no additional value that couldn't be obtained otherwise. As the rationale, the PEP lists: 1. > It is generally a tempting idea to use python as an interface to > a program. I cannot see how this rationale is related to the PEP. You can use Python as interface to a program with or without __source__. 2. > The developers can implement all the functionality > and instead of designing a user interface, provide a python > interpreter to their users. This does not require __source, either. 3. > A much more powerful approach would be an interface which is > dynamically constructed by the user to meet the user's needs. Dynamic code generation doesn't require __source__, either. 4. > The most common development cycle of python programs is: > write .py file - execute .py file - exit - enhance .py file - > execute .py file - etc. With the implementation of the __source__ > attribute though the development/modification of python code > can happen at run-time. This works just fine as well at the moment; see IDLE for an example. > Functions and classes can be defined, > modified or enhanced while the python shell is running and > all the changes can be saved by saving the __source__ attribute > of globals before termination. Currently, you can define classes dynamically, and you can also save the source code of the class to a file in case you need it later. > Moreover, in such a system > it is possible to modify the "code modification routines" and > eventually we have a self-modifying interface. Using a > program also means improving its usability. Self-modifying source code is currently also possible. Just read the old source code from a .py file, modify it, and recompile it. > The current solution of using 'inspect' to get the source > code of functions is not adequate because it doesn't work > for code defined with "exec" and it doesn't have the source > of functions/classes defined in the interactive mode. I fail to see why it isn't adequate. Anybody who wants to modify source code that was originally passed to exec just needs to preserve a copy of the source code, separately. > Generally, > a "file" is something too abstract. What is more real is the > data received by the python parser and that is what is stored > in __source__. Not at all. A file is precisely the level of granularity that is burnt into the Python language. A module is *always* a file, executed from top to bottom. It is not possible to recreate the source code of a module if you have only the source code of all functions, and all classes. Regards, Martin From edcjones at erols.com Fri Dec 3 03:24:45 2004 From: edcjones at erols.com (Edward C. Jones) Date: Sat Dec 4 03:23:03 2004 Subject: [Python-Dev] test_shutils.py fails for 2.4 install Message-ID: <41AFCE6D.10300@erols.com> I have a PC with an AMD cpu and Mandrake 10.1. While installing Python 2.4 "make test" failed in "test_shutils.py". Here is the output of "./python ./Lib/test/test_shutil.py": test_dont_copy_file_onto_link_to_itself (__main__.TestShutil) ... ok test_dont_move_dir_in_itself (__main__.TestShutil) ... ok test_on_error (__main__.TestShutil) ... FAIL test_rmtree_dont_delete_file (__main__.TestShutil) ... ok test_rmtree_errors (__main__.TestShutil) ... ok ====================================================================== FAIL: test_on_error (__main__.TestShutil) ---------------------------------------------------------------------- Traceback (most recent call last): File "./Lib/test/test_shutil.py", line 34, in test_on_error self.assertEqual(self.errorState, 2) AssertionError: 0 != 2 ---------------------------------------------------------------------- Ran 5 tests in 0.005s FAILED (failures=1) Traceback (most recent call last): File "./Lib/test/test_shutil.py", line 106, in ? test_main() File "./Lib/test/test_shutil.py", line 103, in test_main test_support.run_unittest(TestShutil) File "/usr/local/src/Python-2.4/Lib/test/test_support.py", line 290, in run_u run_suite(suite, testclass) File "/usr/local/src/Python-2.4/Lib/test/test_support.py", line 275, in run_s raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "./Lib/test/test_shutil.py", line 34, in test_on_error self.assertEqual(self.errorState, 2) AssertionError: 0 != 2 From michael.walter at gmail.com Sat Dec 4 06:19:20 2004 From: michael.walter at gmail.com (Michael Walter) Date: Sat Dec 4 06:19:23 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <877e9a17041203211849479ab5@mail.gmail.com> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> <877e9a17041203211849479ab5@mail.gmail.com> Message-ID: <877e9a170412032119589f324e@mail.gmail.com> Hi! I think you were the omitting the more interesting closure examples (namely with free variables inside the closure's source): def foo(): pass def bar(x): def fiep(): return x() return fiep what's bar(foo).__source__? Generally, I'm opposed to the feature -- I don't think it provides a real advantage giving it's limitations (doesn't work for import'ed modules, classes + methods != module, etc.). Cheers, Michael On Fri, 3 Dec 2004 11:54:25 +0200 (EET), Stelios Xanthakis <sxanth@ceid.upatras.gr> wrote: > > Hi all. > > Now that 2.4 is out and everything maybe it's > about time to start discussing the "use the > __source__ Luke" feature which IMO will really > boost python into a new domain of exciting > possibilities. > > I've prepared a pre-PEP which is not very good > but it is a base. > > In short, the feature is good and it enables > editing of python code at runtime instead of > the runfile-exit-edit-run-exit-edit-run cycle. > > We have the following possibilities as to whether > __source__ data is marshalled and the feature is > always enabled. > > [1] Command line switch and not marshalled > [2] Always on and not marshalled > [3] Always on and marshalled > > There is also [4] which doesn't make much sense. > > If I was BDFL I'd go for [1] so whoever wants it > can enable it and whoever doesn't can't complain, > and they'll all leave me alone. > Phillip J. Eby expressed some concerns that the > modules that depend on __source__ will eventually > take over and it will become a standard. > > Anyway, the PEP is attached. > You can mail me with votes on the feature and if you > want on your preferred option from 1,2,3. > If I get votes I'll post the results later. > > If this is accepted I'll try to come up with a good > patch vs 2.4 > > Thanks, > > St. > > -------------------ATTACHED PYTHON ENHANCEMENT PROPOSAL--- > PEP: XXX > Title: The __source__ attribute > Version: $Revision: 1.10 $ > Last-Modified: $Date: 2003/09/22 04:51:49 $ > Author: Stelios Xanthakis > Status: Draft > Type: Standards Track > Content-Type: text/plain > Created: 19-Nov-2004 > Python-Version: 2.4.1 > Post-History: > > Abstract > > This PEP suggests the implementation of __source__ attribute for > functions and classes. The attribute is a read-only string which > is generated by the parser and is a copy of the original source > code of the function/class (including comments, indentation and > whitespace). > > Motivation > > It is generally a tempting idea to use python as an interface to > a program. The developers can implement all the functionality > and instead of designing a user interface, provide a python > interpreter to their users. Take for example one of the existing > web browsers: they have everything that would be needed to write > a script which downloads pages automatically or premutates the > letters of web pages before they are displayed, but it is not > possible for the user to do these things because the interface > of these applications is static. > > A much more powerful approach would be an interface which is > dynamically constructed by the user to meet the user's needs. > The most common development cycle of python programs is: > write .py file - execute .py file - exit - enhance .py file - > execute .py file - etc. With the implementation of the __source__ > attribute though the development/modification of python code > can happen at run-time. Functions and classes can be defined, > modified or enhanced while the python shell is running and > all the changes can be saved by saving the __source__ attribute > of globals before termination. Moreover, in such a system > it is possible to modify the "code modification routines" and > eventually we have a self-modifying interface. Using a > program also means improving its usability. > > The current solution of using 'inspect' to get the source > code of functions is not adequate because it doesn't work > for code defined with "exec" and it doesn't have the source > of functions/classes defined in the interactive mode. Generally, > a "file" is something too abstract. What is more real is the > data received by the python parser and that is what is stored > in __source__. > > Specification > > The __source__ attribute is a read-only attribute of functions > and classes. Its type is string or None. In the case of None > it means that the source was not available. > > The indentation of the code block is the original identation > obeying nested definitions. For example: > > >>> class A: > ... def foo (self): > ... print """Santa-Clauss > ... is coming to town""" > >>> def spam (): > ... def closure (): > ... pass > ... return closure > >>> print A.foo.__source__ > def foo (self): > print """Santa-Clauss > is coming to town""" > >>> print spam().__source__ > def closure (): > pass > > The attribute is not marshaled and therefore not stored in > ".pyc" files. As a consequence, functions and classes of > imported modules have __source__==None. > > We propose that the generation of __source__ will be > controlled by a command line option. In the case this > feature is not activated by the command line option, the > attribute is absent. > > Rationale > > Generally, "import" refers to modules that either have a file in > a standard location or they are distributed in ".pyc" form only. > Therefore in the case of modules, getting the source with > "inspect" is adequate. Moreover, it does not make sense saving > __source__ in ".pyc" because the point would be to save > modifications in the original ".py" file (if available). > > On the issue of the command-line option controlling the generation > of __source__, please refer to the section about the overhead > of this feature. The rationale is that those applications that > do not wish to use this feature can avoid it (cgi scripts in > python benchmarked against another language). > > Overhead > > The python's parser is not exactly well-suited for such a feature. > Execution of python code goes through the stages of lexical > analysis, tokenization, generation of AST and execution of > bytecode. In order to implement __source__, the tokenizer has > to be modified to store the lines of the current translation > unit. Those lines are then attached the root node of the > AST. While the AST is compiled we have to keep a reference > of the current node in order to be able to find the next node > after the node for which we wish to generate __source__, get > the first and the last line of our block and then refer to > the root node to extract these lines and make a string. All > these actions add a minor overhead to some heavily optimized > parts of python. However, once compilation to bytecode is > done, this feature no longer affects the performance of the > execution of the bytecode. > > There is also the issue of the memory spent to store __source__. > In our opinion, this is worth the tradeoff for those who > are willing to take advantage of it. > > Implementation > > There is a sample implementation at [2] which consists of a > patch against python 2.3.4. The patch has to be improved > to avoid generating __source__ for the case we are importing > modules for the first time (not from .pyc). In the sample > implementation there is also included a sample shell that > takes advantage of __source__ and demonstrates some aspects > that motivated us towards patching python and submitting this > PEP. > > References > > [1] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton > http://www.python.org/peps/pep-0001.html > > [2] Sample implementation > http://students.ceid.upatras.gr/~sxanth/ISYSTEM/python-PIESS.tar.gz > > Copyright > > This document has been placed in the public domain. > > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > End: > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com > From hpk at trillke.net Sat Dec 4 09:59:38 2004 From: hpk at trillke.net (holger krekel) Date: Sat Dec 4 09:59:41 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <Pine.GSO.4.61.0412031554001.5550@zenon.ceid.upatras.gr> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> <20041203123616.GB4708@solar.trillke.net> <Pine.GSO.4.61.0412031554001.5550@zenon.ceid.upatras.gr> Message-ID: <20041204085937.GC4708@solar.trillke.net> [Stelios Xanthakis Fri, Dec 03, 2004 at 11:59:30PM +0200] > On Fri, 3 Dec 2004, holger krekel wrote: > >We are about to test out this approach with the py lib > >(http://codespeak.net/py) and want to have it work for > >for Python 2.2, 2.3. and 2.4. > > Do you plan hacking python ? > It appears that tok_nextc() is the best place to > catch all the source passed to the interpreter. Well, as we want to have the library work on past python versions modifying CPython 2.5 does not make much sense. It's more about (like Martin pointed out) organizing dynamic code generation so that Python's introspect and traceback logic works as much as possible - with tiny runtime "monkey" patches if needed. Now Martin also correctly pointed out that you can store source code before/after you pass it to compile/parse. We are doing this already with an external dictionary. This has multithreading issues, though. So we think that hooking onto code's objects co_filename or a module's __file__ might be an interesting idea. cheers, holger From sxanth at ceid.upatras.gr Sat Dec 4 10:02:26 2004 From: sxanth at ceid.upatras.gr (Stelios Xanthakis) Date: Sat Dec 4 10:02:30 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <41B0F629.4040100@v.loewis.de> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> <41B0F629.4040100@v.loewis.de> Message-ID: <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr> > I'm opposed to this idea. It creates overhead in > the size of .pyc files, No it doesn't. > for no additional value that couldn't be obtained otherwise. Martin: I know it is possible to do all this with existing python facilities. I did write such a dynamic code framework in python. Specifically I used a function 'deyndef(code)' which was exactly like 'def' but also stored the source string in a dictionary. The key point is that I think think should be the job of the parser and the functionality provided at the interactive prompt w/o the user having to write 'dyndef' or store the code of exec's or request from himself to use specific commands to create functions. It should be transparent built into python. > A file is precisely the level of granularity that is > burnt into the Python language. A module is *always* a file, executed > from top to bottom. It is not possible to recreate the source code > of a module if you have only the source code of all functions, and > all classes. That's exactly the rationale for NOT combining __source__ with import. It's in the PEP. It appears that there are the 'module people' who find this feature irrelevant. Indeed. If we are interested in distributing modules and increasing the number of people who use python programs,then __source__ is redundant. OTOH, programming python is easy and fun and I think the proposed feature will make it even more fun and it aims in increasing the number of people who program python for their every day tasks. It'd be interesting to hear if the developers of IDLE/ipython/etc could use this. Oh well. I guess I'm ahead of my time again:) St. From mwh at python.net Sat Dec 4 12:57:34 2004 From: mwh at python.net (Michael Hudson) Date: Sat Dec 4 12:57:35 2004 Subject: [Python-Dev] test_shutils.py fails for 2.4 install In-Reply-To: <41AFCE6D.10300@erols.com> (Edward C. Jones's message of "Thu, 02 Dec 2004 21:24:45 -0500") References: <41AFCE6D.10300@erols.com> Message-ID: <2m7jnys2c1.fsf@starship.python.net> "Edward C. Jones" <edcjones@erols.com> writes: > I have a PC with an AMD cpu and Mandrake 10.1. While installing Python > 2.4 "make test" failed in "test_shutils.py". Here is the output of > "./python ./Lib/test/test_shutil.py": Are you running 'make test' as root? Don't do that (although possibly the test suite should guard against it). Also, did you search the bug tracker? Please do do that. Cheers, mwh -- <glyph> CDATA is not an integration strategy. -- from Twisted.Quotes From martin at v.loewis.de Sat Dec 4 15:10:06 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 4 15:10:03 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> <41B0F629.4040100@v.loewis.de> <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr> Message-ID: <41B1C53E.9050204@v.loewis.de> Stelios Xanthakis wrote: > The key point is that I think think should be the > job of the parser and the functionality provided > at the interactive prompt w/o the user having to > write 'dyndef' or store the code of exec's or request > from himself to use specific commands to create > functions. It should be transparent built into python. For the case of the interactive prompt, if preserving the source code is somehow desirable (which I doubt), it should be the job of the development environment to offer saving interactively-defined methods. Such IDE support is necessary even if __source__ was available, since users probably would not want to write open("demo.py").write(myfunc.__source__ + "\n" + myclass.__source) > OTOH, programming python is easy and fun and I think > the proposed feature will make it even more fun and it > aims in increasing the number of people who program > python for their every day tasks. It'd be interesting to > hear if the developers of IDLE/ipython/etc could use this. I think it is the other way 'round. If this is *only* for interactive mode, than you should *first* change the interactive environments. If you then find you absolutely need this feature to implement the IDEs correctly, I'd like to hear the (new) rationale for the change. Regards, Martin From ncoghlan at iinet.net.au Sat Dec 4 15:29:27 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Dec 4 15:29:34 2004 Subject: [Python-Dev] PEP: __source__ proposal In-Reply-To: <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr> References: <Pine.GSO.4.61.0412031128470.4145@zenon.ceid.upatras.gr> <41B0F629.4040100@v.loewis.de> <Pine.GSO.4.61.0412041011200.6772@zenon.ceid.upatras.gr> Message-ID: <41B1C9C7.1020604@iinet.net.au> [Resend, since a minor brain explosion caused me to send this to c.l.p instead of python-dev] Stelios Xanthakis wrote: > It appears that there are the 'module people' who > find this feature irrelevant. Indeed. If we are interested > in distributing modules and increasing the number of > people who use python programs,then __source__ is redundant. > OTOH, programming python is easy and fun and I think > the proposed feature will make it even more fun and it > aims in increasing the number of people who program > python for their every day tasks. It'd be interesting to > hear if the developers of IDLE/ipython/etc could use this. The feedback here (and the initial response on py-dev a while back) suggests to me that you should look at making this a feature of the interactive mode. Something that affects both Python's main interactive shell, plus the relevant class in the standard library (CommandInterpreter or whatever it is called). A late-night-train-of-thought example of what might be handy is below - keep in mind that I haven't looked at what enhanced Python shells like IPython can do, so it may be there are tools out there that do something like this already. It would be handy to have a standard library module that supported "on-the-fly" editing, though (this capability would then be available to anyone embedding Python as a scripting engine). Cheers, Nick. ============================== >>>import source >>>class bob: ... def mary(): ... pass ... def tim(): ... print 'Tim' ... >>>print bob.__source__ class bob: def mary(): pass def tim(): print 'Tim' >>>print bob.mary.__source__ def mary(): pass >>> source.edit(bob.mary) bob.mary(1)>def mary(text): # [1] bob.mary(2)> print "Mary:", text bob.mary(3)>\save >>> source.edit(bob.tim) bob.tim(1)>\help Commands: \help \cancel \save \deleteline bob.tim(2)>\cancel >>>print bob.__source__ "class bob: def mary(text): print "Mary:", text def tim(): print 'Tim' " >>> bob().mary("Hi!") Mary: Hi! The basic ideas of the above: "import source" triggers the storage of the __source__ attributes (e.g. via installation of appropriate hooks in the class and function definition process) The 'edit' function is then able to take advantage of the stored source code to present each line of the original source for modification (e.g. to fix a minor bug in one function of a class definition). When the 'edit' is complete, it can be saved or cancelled. 1. The feature mentioned in the last paragraph is hard to show in the expected output :) -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From tim.peters at gmail.com Sat Dec 4 17:23:26 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Dec 4 17:23:30 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib whrandom.py, 1.21, NONE In-Reply-To: <E1CaXV7-0004ui-Sy@sc8-pr-cvs1.sourceforge.net> References: <E1CaXV7-0004ui-Sy@sc8-pr-cvs1.sourceforge.net> Message-ID: <1f7befae04120408235ffc9b8a@mail.gmail.com> [rhettinger@users.sourceforge.net] > Removed Files: > whrandom.py > Log Message: > Remove the deprecated whrandom module. Woo hoo! It's about friggin' time <wink>. From edcjones at erols.com Fri Dec 3 18:53:03 2004 From: edcjones at erols.com (Edward C. Jones) Date: Sat Dec 4 18:51:20 2004 Subject: [Python-Dev] RE: test_shutils.py fails for 2.4 install Message-ID: <41B0A7FF.9040305@erols.com> The relevant bug appears to be 1076467 at SourceForge. From raymond.hettinger at verizon.net Sun Dec 5 03:20:09 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun Dec 5 03:22:54 2004 Subject: [Python-Dev] Py2.5: Deprecated Cookie classes and interface Message-ID: <000901c4da70$f2201e40$ab2fc797@oemcomputer> Any objections to removing Cookie.Cookie, Cookie.SmartCookie, and Cookie.SerialCookie? Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20041204/e3c8679b/attachment.html From raymond.hettinger at verizon.net Sun Dec 5 03:30:03 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun Dec 5 03:32:47 2004 Subject: [Python-Dev] Deprecated xmllib module Message-ID: <001801c4da72$543801a0$ab2fc797@oemcomputer> Hmph. The xmllib module has been deprecated since Py2.0 but is not listed in PEP 4. Question: Do we have to keep it for another two years because of that omission? It seems somewhat silly to keep an obsolete, supplanted module that doesn?t full support XML 1.0. Raymond From fredrik at pythonware.com Sun Dec 5 08:40:47 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun Dec 5 08:38:34 2004 Subject: [Python-Dev] Re: Deprecated xmllib module References: <001801c4da72$543801a0$ab2fc797@oemcomputer> Message-ID: <coudtk$ef$1@sea.gmane.org> Raymond Hettinger wrote: > Hmph. The xmllib module has been deprecated since Py2.0 but is not > listed in PEP 4. > > Question: Do we have to keep it for another two years because of that > omission? > > It seems somewhat silly to keep an obsolete, supplanted module that > doesn't full support XML 1.0. it's better to be somewhat silly than to be arrogant and stupid. if python-dev cares about existing users, xmllib should stay in there until 3.0. </F> From bac at OCF.Berkeley.EDU Sun Dec 5 08:52:14 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Dec 5 08:52:21 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? Message-ID: <41B2BE2E.3000907@ocf.berkeley.edu> I noticed that Makefile.pre.in uses the value from the environment variable LDFLAGS but not CPPFLAGS. Any reason for this? ``./configure -h`` lists both (and some other environment variables which are not really used either) so it seems a little misleading to have those listed but to not utilize them. The reason I ask is I plan on having setup.py add the directories specified in both of these environment variables for compiling the extension modules. It would be nice to be able to use the same values as used by the Makefile to build the core, but I can if I must just get the values directly from the environment variables themselves. This should allow for the removal of the direct support for Fink and DarwinPorts. It should also remove any hand-editing needed to get setup.py to pick up any non-standard library and header locations. -Brett From martin at v.loewis.de Sun Dec 5 09:36:00 2004 From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 5 09:35:59 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <001801c4da72$543801a0$ab2fc797@oemcomputer> References: <001801c4da72$543801a0$ab2fc797@oemcomputer> Message-ID: <41B2C870.5090609@v.loewis.de> Raymond Hettinger wrote: > Hmph. The xmllib module has been deprecated since Py2.0 but is not > listed in PEP 4. > > Question: Do we have to keep it for another two years because of that > omission? > > It seems somewhat silly to keep an obsolete, supplanted module that > doesn?t full support XML 1.0. I mostly agree with Fredrik. What good does removal of xmllib do? It's not that it is causing any maintenance burden, so we could just leave it in. Whether this means that we keep xmllib until P3k, I don't know. As for PEP 4: I don't know whether it needs to be listed there. It appears that the PEP is largely unmaintained (I, personally, do not really maintain it). So one option would be to just stop using PEP 4 for recording deprecations, since we now have the warnings module. If we want to keep PEP 4, we need to follow the procedures it requires (or modify them if we don't like them). Regards, Martin From martin at v.loewis.de Sun Dec 5 09:40:49 2004 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun Dec 5 09:40:45 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B2BE2E.3000907@ocf.berkeley.edu> References: <41B2BE2E.3000907@ocf.berkeley.edu> Message-ID: <41B2C991.4090502@v.loewis.de> Brett C. wrote: > I noticed that Makefile.pre.in uses the value from the environment > variable LDFLAGS but not CPPFLAGS. Any reason for this? How did you notice that? For LDFLAGS, Makefile.pre.in has LDFLAGS= @LDFLAGS@ This does *not* mean that the value from the environment is used. Instead, it means that configure computes the value of LDFLAGS when it generates Makefile.in. For CPPFLAGS, configure has nothing to compute, so Makefile.pre.in just has the static value for CPPFLAGS. Regards, Martin From python at rcn.com Sun Dec 5 10:27:15 2004 From: python at rcn.com (Raymond Hettinger) Date: Sun Dec 5 10:30:03 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B2C870.5090609@v.loewis.de> Message-ID: <000201c4daac$9bd871e0$3b01a044@oemcomputer> > As for PEP 4: I don't know whether it needs to be listed there. It > appears that the PEP is largely unmaintained (I, personally, do not > really maintain it). So one option would be to just stop using PEP 4 > for recording deprecations, since we now have the warnings module. +1 Raymond From mwh at python.net Sun Dec 5 11:40:08 2004 From: mwh at python.net (Michael Hudson) Date: Sun Dec 5 11:40:10 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B2BE2E.3000907@ocf.berkeley.edu> (Brett C.'s message of "Sat, 04 Dec 2004 23:52:14 -0800") References: <41B2BE2E.3000907@ocf.berkeley.edu> Message-ID: <2my8gdqb93.fsf@starship.python.net> "Brett C." <bac@OCF.Berkeley.EDU> writes: > I noticed that Makefile.pre.in uses the value from the environment > variable LDFLAGS but not CPPFLAGS. Any reason for this? > ./configure -h`` lists both (and some other environment variables > which are not really used either) so it seems a little misleading to > have those listed but to not utilize them. The whole story of how the compiling/linker flags get set up is a bit of a mess, AFAIK. I don't have the energy or the desire to try to make what we want to happen precise (the hard bit) and then make that happen (probably rather easier). > This should allow for the removal of the direct support for Fink and > DarwinPorts. +lots. I don't trust fink, but need it for latex... Cheers, mwh -- After a heavy night I travelled on, my face toward home - the comma being by no means guaranteed. -- paraphrased from cam.misc From python at rcn.com Sun Dec 5 11:40:16 2004 From: python at rcn.com (Raymond Hettinger) Date: Sun Dec 5 11:43:06 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B2C870.5090609@v.loewis.de> Message-ID: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> > > It seems somewhat silly to keep an obsolete, supplanted module that > > doesn?t full support XML 1.0. > > I mostly agree with Fredrik. What good does removal of xmllib do? A few thoughts: * Deprecated modules just get moved to the lib-old directory. If someone has ancient code relying on the module, it is a somewhat trivial maintenance step to add lib-old to their PYTHONPATH. IOW, I fail to see the harm. * For this particular module, xmllib, about six years will have elapsed between its original deprecation in Py2.0 and us taking it out in a Py2.5 release. * The number one current python complaint is the state of the standard library: http://www.internetnews.com/dev-news/article.php/3441931 . Some of this may just be the fruits of AMK's highly publicized journal entry; however, I think the concerns about library quality will persist. * The average quality of the library improves as we take out junk (the tzparse module for example) and put in high quality modules like logging, csv, decimal, etc. * After working through the tutorial, another huge task for aspiring intermediate Python programmer is to learn about the standard library. That task is made much difficult by the presence of obsolete, incomplete, and/or supplanted modules. For example, the high quality, actively maintained email package supplants several older, incomplete, or non-actively maintained modules. The library would be cleaner, more presentable, and easier to use if the old ones were to fall by the wayside. * Ideally, the library will develop in a way that doesn't have a cluttered concept space. This will help get it back to the point where you can hold it in your head all at once and still be able to think about the problem domain. Keeping cruft impedes that goal. my two cents, Raymond From martin at v.loewis.de Sun Dec 5 12:12:12 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 5 12:12:08 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> Message-ID: <41B2ED0C.20509@v.loewis.de> Raymond Hettinger wrote: > * Deprecated modules just get moved to the lib-old directory. If > someone has ancient code relying on the module, it is a somewhat trivial > maintenance step to add lib-old to their PYTHONPATH. IOW, I fail to see > the harm. I have never considered this as an official policy. For example, when deprecated C modules are removed, they are never moved to lib-old. > * For this particular module, xmllib, about six years will have elapsed > between its original deprecation in Py2.0 and us taking it out in a > Py2.5 release. Correct. For regex, much more time has passed (it is deprecated since 1.5 or something). > * The number one current python complaint is the state of the standard > library: http://www.internetnews.com/dev-news/article.php/3441931 . > Some of this may just be the fruits of AMK's highly publicized journal > entry; however, I think the concerns about library quality will persist. I agree. Removing old modules might change this, indeed: the complaint about unstability and incompatibility might then become the number one complaint :-) > * The average quality of the library improves as we take out junk (the > tzparse module for example) and put in high quality modules like > logging, csv, decimal, etc. I am not convinced that all these more recent modules are really higher quality than the modules that have been added ten years ago. Some are (especially those which has seen extensive testing before being integrated), some are not (especially those added in an ad-hoc manner). Regards, Martin From anthony at interlink.com.au Sun Dec 5 16:42:56 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Sun Dec 5 16:43:28 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> Message-ID: <200412060243.01129.anthony@interlink.com.au> On Sunday 05 December 2004 21:40, Raymond Hettinger wrote: > * The number one current python complaint is the state of the standard > library: http://www.internetnews.com/dev-news/article.php/3441931 . > Some of this may just be the fruits of AMK's highly publicized journal > entry; however, I think the concerns about library quality will persist. > > * The average quality of the library improves as we take out junk (the > tzparse module for example) and put in high quality modules like > logging, csv, decimal, etc. We can't win - if we remove it, we'll get massive numbers of complaints bitching about us breaking backwards compatibility. I think the solution is to make sure the library docs say, very clearly, in big words at the top of the relevant page "DON'T DO NEW CODE WITH THIS MODULE". Perhaps Fred can make latex2html output a <blink> tag around it <wink>. The library docs for, e.g. xmllib already say deprecated at the top - maybe it should be larger? Should the global module index should grow a "(deprecated)" line next door to the module name of the modules that are known to be old and kinda awful (such as xmllib)? Or make a seperate category - normal modules, and modules-that-are-only-kept-for-backwards compat? I suspect that rfc822 will end up in that latter category, rather than ever being removed (at least, until 3.0). I went through just my own code, and found an amazing amount of use of that module. Python has a reputation for stability and not forcing people to rewrite their code - I think that this is a good thing. IOW, I think breaking code is far worse than some folks whining that some of the stdlib is old and crufty. Alternately, we could lock the folks complaining about the stdlib's state in a room with the folks who complain that every new thing has "wrecked the language that they knew and loved" and let them fight it out. > * After working through the tutorial, another huge task for aspiring > intermediate Python programmer is to learn about the standard library. > That task is made much difficult by the presence of obsolete, > incomplete, and/or supplanted modules. Surely this one is something that can be fixed in the documentation by marking obsolete modules as such, or making the marking much clearer? Anthony From barry at python.org Sun Dec 5 17:35:13 2004 From: barry at python.org (Barry Warsaw) Date: Sun Dec 5 17:35:17 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B2C870.5090609@v.loewis.de> References: <001801c4da72$543801a0$ab2fc797@oemcomputer> <41B2C870.5090609@v.loewis.de> Message-ID: <1102264513.29521.63.camel@presto.wooz.org> On Sun, 2004-12-05 at 03:36, "Martin v. L?wis" wrote: > I mostly agree with Fredrik. What good does removal of xmllib do? > It's not that it is causing any maintenance burden, so we could just > leave it in. Whether this means that we keep xmllib until P3k, I > don't know. > > As for PEP 4: I don't know whether it needs to be listed there. It > appears that the PEP is largely unmaintained (I, personally, do not > really maintain it). So one option would be to just stop using PEP 4 > for recording deprecations, since we now have the warnings module. > If we want to keep PEP 4, we need to follow the procedures it > requires (or modify them if we don't like them). I agree. We don't need to use PEP 4 for listing module deprecation schedules. It would be better if we included that information in the DeprecationWarning. Probably nobody reads PEP 4 when they get a DeprecationWarning, but everyone reads the message that's printed, so if it said something like "This module is deprecated and slated for removal in Python 2.6", I think we'd be providing better information to our users. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041205/094182eb/attachment.pgp From barry at python.org Sun Dec 5 17:50:22 2004 From: barry at python.org (Barry Warsaw) Date: Sun Dec 5 17:50:25 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> Message-ID: <1102265422.29524.74.camel@presto.wooz.org> On Sun, 2004-12-05 at 05:40, Raymond Hettinger wrote: > * The number one current python complaint is the state of the standard > library: http://www.internetnews.com/dev-news/article.php/3441931 . > Some of this may just be the fruits of AMK's highly publicized journal > entry; however, I think the concerns about library quality will persist. I don't know if I'd use one quote from this article to infer that standard library issues are the #1 complaint. But besides that, there's no question IMO that the standard library should be the place where future Python development efforts are focused (not exclusively, but predominately). > * The average quality of the library improves as we take out junk (the > tzparse module for example) and put in high quality modules like > logging, csv, decimal, etc. > > * After working through the tutorial, another huge task for aspiring > intermediate Python programmer is to learn about the standard library. > That task is made much difficult by the presence of obsolete, > incomplete, and/or supplanted modules. You don't have to remove modules to make this happen. Few developers find modules by doing an 'ls' on Lib -- they look to the library reference docs, books, and existing code. So for example, the re-org I did on the string module docs for 2.4 should help, because now you can't look up a string function such as atoi() without seeing that it's deprecated. A developer writing new code hopefully wouldn't even waste time learning about those functions, and instead hop right to the preferred string methods. Yet, there's no push in the community to actually remove those deprecated functions from string.py. > For example, the high quality, actively maintained email package > supplants several older, incomplete, or non-actively maintained modules. > The library would be cleaner, more presentable, and easier to use if the > old ones were to fall by the wayside. Yes, but there's still /a lot/ of work to do to update the existing standard library to use the email package. We debated this during 2.4 and decided we can't even add deprecation warnings to things like rfc822.py until we do. > * Ideally, the library will develop in a way that doesn't have a > cluttered concept space. This will help get it back to the point where > you can hold it in your head all at once and still be able to think > about the problem domain. Keeping cruft impedes that goal. Agreed, but I think it's mostly going to be an organic process over time. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041205/71ac1d23/attachment.pgp From carribeiro at gmail.com Sun Dec 5 18:42:45 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Sun Dec 5 18:42:48 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> References: <41B2C870.5090609@v.loewis.de> <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> Message-ID: <864d370904120509421ec3dd50@mail.gmail.com> On Sun, 5 Dec 2004 05:40:16 -0500, Raymond Hettinger <python@rcn.com> wrote: > * Deprecated modules just get moved to the lib-old directory. If > someone has ancient code relying on the module, it is a somewhat trivial > maintenance step to add lib-old to their PYTHONPATH. IOW, I fail to see > the harm. In principle, that's a good idea, but I don't know if it's really practical. I fear that the distribution would end up simply adding lib-old to the PYTHONPATH anyway :-) But some variations may be worth considering: 1) Deprecated modules would first generate a warning (as it is today). On future versions, instead of being removed, the level of the warning would be raised, causing a fatal exception (an ImportError?), unless explicitly configured otherwise (using a config file or some similar mechanism). I believe that the current warnings module already provides a good basis for this implementation. (IOW - the module would still be there, and could be activated, but it's "off" by default. Turning it on again has to be easy, though. And the message on "ImportError" have to meaningful for non-technical users, allowing for easier support in these situations) 2) Split the documentation: the "old-and-deprecated" modules would be listed separated from the active & recommended ones, as a different section, or even as a separate book. That would be a clear sign to new users to keep in mind while they read the documentation, perhaps more effective than writing the deprecation warning on the top of the page. My half-a-cent. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From martin at v.loewis.de Sun Dec 5 18:54:42 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 5 18:54:39 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <200412060243.01129.anthony@interlink.com.au> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> Message-ID: <41B34B62.7040201@v.loewis.de> Anthony Baxter wrote: > The library docs for, e.g. xmllib already say deprecated at the top - maybe > it should be larger? I think it would be better to *remove* the xmllib documentation. Existing code which needs the module will continue to work even without documentation, and new code is unlikely to be written for a module that has no documentation, and where importing the module gives a DeprecationWarning. > Alternately, we could lock the folks complaining about the stdlib's state > in a room with the folks who complain that every new thing has "wrecked > the language that they knew and loved" and let them fight it out. That sounds like a fair and democratic way of solving the issue. Regards, Martin From allison at sumeru.stanford.EDU Sun Dec 5 19:30:28 2004 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Sun Dec 5 19:30:35 2004 Subject: [Python-Dev] Python 2.4 and Zope 2.7.X In-Reply-To: <1102265422.29524.74.camel@presto.wooz.org> Message-ID: <Pine.LNX.4.10.10412051027310.4966-100000@sumeru.stanford.EDU> A report on the zope@zope.org list suggests that Python 2.4 is not fully compatible with Zope 2.7.3. Has any tested against Zope? From lists at andreas-jung.com Sun Dec 5 21:06:26 2004 From: lists at andreas-jung.com (Andreas Jung) Date: Sun Dec 5 21:06:33 2004 Subject: [Python-Dev] Python 2.4 and Zope 2.7.X In-Reply-To: <Pine.LNX.4.10.10412051027310.4966-100000@sumeru.stanford.EDU> References: <Pine.LNX.4.10.10412051027310.4966-100000@sumeru.stanford.EDU> Message-ID: <17891C1A2E0C2C4C4580A33B@[192.168.0.102]> --On Sonntag, 5. Dezember 2004 10:30 Uhr -0800 Dennis Allison <allison@sumeru.stanford.EDU> wrote: > > A report on the zope@zope.org list suggests that Python 2.4 is not fully > compatible with Zope 2.7.3. Has any tested against Zope? > To which report do you refer? The only thing I mentioned is that there has not been any audit for Python 2.4 + Zope 2.7 as with Python 2.3 + Zope 2.6/2.7. -aj From rwgk at cci.lbl.gov Mon Dec 6 00:37:12 2004 From: rwgk at cci.lbl.gov (Ralf W. Grosse-Kunstleve) Date: Mon Dec 6 00:37:19 2004 Subject: [Python-Dev] proposal+patch: sys.gettickeraccumulation() Message-ID: <200412052337.iB5NbC5Y318887@boa.lbl.gov> Brett C. wrote: > Ralf W. Grosse-Kunstleve wrote: > > Brett C. wrote: > > > > >I have been mulling over this proposal and I think I am finally > > >settling on +0 as long as it can be shown that it does not hurt > > >performance at all (yes, it shouldn't but this is the eval loop we are > > >talking about and that thing is a touchy beast). > > > > > > Is there a standard benchmark I could use to demonstrate the > > impact of the two new additions on the runtime? > > Thanks! > > =) Parrotbench and PyBench would be enough for me to sign off on any > performance hit. There is also pystone. I have done my homework now, based on parrotbench. Timings are below. The most important conclusion from my viewpoint is that my original goal can be achieved with a small patch and with a runtime penalty that is in the noise. However, mysterious runtime behavior of Objects/longobject.c lead me to change my original patch, and to the unanswered question "could small rearrangements in longobject.c potentially boost performance by 45% on some platforms?" Please don't be put off by the size of this message. My patch is essentially just one line in ceval.c (_Py_TickerAccumulation++)! To restate my original goal: I am looking for a simple way to answer the question: How much of a speedup can I expect if I reimplement a piece of Python code in C or C++? Note that I am not asking how much time is spent in Python including all the extensions that come with it, but how much time is spent in the Python bytecode interpreter loop. To clarify, look at the timings for two tests in parrotbench: b1 5.8287 time 30049474 ticks 0.1940 us/tick b2 1.5944 time 482584 ticks 3.3039 us/tick Each of the tests was run with range(10) in b.py to increase accuracy. The first column shows time.time() differences, the second the number of bytecodes processed in ceval.c, and the last column shows micro-seconds per tick. Why is the us/tick value for b2 17 times larger than that for b1? The answer is that b2 makes heavy use of long integers, and that a lot of time is spent in Objects/longobject.c doing long-integer arithmetic. Compared to b1, relatively little time is spent interpreting bytecodes. Back to the original question: how much sense does it make to reimplement b1 or b2 in C? Simply looking at the us/tick values is telling me that I can expect much more of a performance boost by reimplementing b1 rather than b2. This is because b2 spends a lot of time in C already. A similar situation arises if extensions like Numeric are used. Computationally intensive operations like matrix multiplication and matrix inversion are already done at C speed. The us/tick let us quickly estimate how disappointed we would be after moving Python code to C or C++. We get this estimate without having to do detailed source code analysis, and before we have wasted our time doing the recoding (we have been through this a couple of times ...). The timings below show that the patched Python which counts the number of interpreted bytecodes is in the worst case 0.6% slower than the original Python, but some tests even run faster at face value: $gcc296_compiled/original/python -O b_timings.py | grep reporter b0 16.8831 time b1 5.9553 time b2 1.5914 time b3 10.8149 time b4 5.2528 time b5 11.6437 time b6 11.0248 time b7 27.6960 time all 90.9276 time $gcc296_compiled/work/python -O b_timings.py | grep reporter b0 16.9193 time 50979585 ticks 0.3319 us/tick b1 5.8287 time 30049474 ticks 0.1940 us/tick b2 1.5944 time 482584 ticks 3.3039 us/tick b3 10.7931 time 43577404 ticks 0.2477 us/tick b4 5.2865 time 18463044 ticks 0.2863 us/tick b5 11.6086 time 17065750 ticks 0.6802 us/tick b6 10.0992 time 60000464 ticks 0.1683 us/tick b7 27.6830 time 1928644 ticks 14.3536 us/tick all 89.8760 time 222548399 ticks 0.4038 us/tick The b7 test is the same as b2 but with xrange(10000) instead of xrange(1000), and with all print statements removed. Ratios (rounded to 3 decimals): 16.9193/16.8831=1.002 5.8287/5.9553 =0.979 1.5944/1.5914 =1.002 10.7931/10.8149=0.998 5.2865/5.2528 =1.006 11.6086/11.6437=0.997 10.0992/11.0248=0.916 27.6830/27.6960=1.000 Therefore I'd argue that the runtime penalty for the one additional long long increment in ceval.c (_Py_TickerAccumulation++) is in the noise. The timings were collected on a 2.8GHz Dual Xeon, Redhat WS 3. Python was compiled under Red Hat 7.3 with gcc 2.96. See below why. My latest patch can be found at the end of this posting. It can be applied to the Python-2.4 distribution like this: tar zxf Python-2.4.tgz patch --strip=1 -d Python-2.4 < patch_file ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ Now to the mysterious runtime behavior of Objects/longobject.c In the original Python 2.4 sources _Py_Ticker is decremented in two places: In the interpreter loop in Python/ceval.c: if (--_Py_Ticker < 0) { In Objects/longobject.c: #define SIGCHECK(PyTryBlock) \ if (--_Py_Ticker < 0) { \ _Py_Ticker = _Py_CheckInterval; \ if (PyErr_CheckSignals()) { PyTryBlock; } \ } This macro is used in four places. My initial patch was to insert updates of a new, global _Py_TickerAccumulation variable in these two places: Python/ceval.c: a try: finally: block uninterruptable. */ goto fast_next_opcode; } + _Py_TickerAccumulation += _Py_CheckInterval - _Py_Ticker; _Py_Ticker = _Py_CheckInterval; tstate->tick_counter++; Objects/longobject.c: #define SIGCHECK(PyTryBlock) \ if (--_Py_Ticker < 0) { \ + _Py_TickerAccumulation += _Py_CheckInterval - _Py_Ticker; \ _Py_Ticker = _Py_CheckInterval; \ if (PyErr_CheckSignals()) { PyTryBlock; } \ } This lead to the timings below. All timings were collected on the same machine, but the original and the patched Pythons were each compiled with two different compilers, gcc 2.96 and gcc 3.2.3. The important timings to look at are those for b2 and b7 again. $gcc296_compiled/original/python -O b_timings.py | grep reporter b0 16.8831 time b1 5.9553 time b2 1.5914 time b3 10.8149 time b4 5.2528 time b5 11.6437 time b6 11.0248 time b7 27.6960 time all 90.9276 time $gcc323_compiled/original/python -O b_timings.py | grep reporter b0 18.6390 time b1 6.4681 time b2 2.2588 time b3 11.0215 time b4 5.6490 time b5 12.3022 time b6 10.3815 time b7 40.2735 time all 107.0636 time This shows that the gcc 2.96 optimizer is generally a little bit better than the gcc 3.2.3 optimizer, but not by very much. Except for the b2 and b7 tests. E.g. 40.2735/27.6960 = 1.454! Now with the patches in both ceval.c and longobject.c: $gcc296_compiled/ticker/python -O b_timings.py | grep reporter b0 17.1190 time 60792625 ticks 0.2816 us/tick b1 6.1171 time 30049474 ticks 0.2036 us/tick b2 1.8764 time 705754 ticks 2.6587 us/tick b3 10.7070 time 43577404 ticks 0.2457 us/tick b4 5.2677 time 22944694 ticks 0.2296 us/tick b5 11.7448 time 17433190 ticks 0.6737 us/tick b6 10.9603 time 60000504 ticks 0.1827 us/tick b7 33.2320 time 2837124 ticks 11.7133 us/tick all 97.0893 time 238342235 ticks 0.4074 us/tick $gcc323_compiled/ticker/python -O b_timings.py | grep reporter b0 18.3115 time 60792625 ticks 0.3012 us/tick b1 6.1193 time 30049474 ticks 0.2036 us/tick b2 2.2522 time 705754 ticks 3.1912 us/tick b3 11.2202 time 43577404 ticks 0.2575 us/tick b4 5.4596 time 22944694 ticks 0.2379 us/tick b5 11.8197 time 17433190 ticks 0.6780 us/tick b6 10.7407 time 60000504 ticks 0.1790 us/tick b7 40.2397 time 2837124 ticks 14.1833 us/tick all 106.2283 time 238342235 ticks 0.4457 us/tick The gcc 3.2.3 timings for b7 are hardly affected by my patch in longobject.c, but the gcc 2.96 timing shoots up from 27.6960 to 33.2320. Still not as bad as the gcc 3.2.3 timing, but why? Interestingly, if I declare _Py_TickerAccumulation as long instead of PY_LONG_LONG, the gcc 2.96 timing is also hardly affected by my patch in longobject.c. Even more interestingly, if I change longobject.c like this... #define SIGCHECK(PyTryBlock) \ { \ if (PyErr_CheckSignals()) { PyTryBlock; } \ } the runtime is also hardly affected (times not shown in this posting) even though PyErr_CheckSignals() is executed 100 times more often. Curious to learn how other compilers deal with longobject.c I collected timings with this compiler: Compaq C V6.3-028 on Compaq Tru64 UNIX V5.1 (Rev. 732) $compaq_compiled/original/python -O b_timings.py | grep reporter b1 19.4492 time b2 4.9395 time b3 23.6787 time b5 28.2568 time b6 21.6230 time b7 86.5283 time all 192.0498 time $compaq_compiled/ticker/python -O b_timings.py | grep reporter b1 19.4775 time 30049474 ticks 0.6482 us/tick b2 4.9971 time 704784 ticks 7.0902 us/tick b3 24.2559 time 43577404 ticks 0.5566 us/tick b5 26.3320 time 17420746 ticks 1.5115 us/tick b6 21.7227 time 60000464 ticks 0.3620 us/tick b7 86.9395 time 2836154 ticks 30.6540 us/tick all 190.8936 time 165555937 ticks 1.1530 us/tick The b0 and b4 timings are missing because of exceptions that I didn't investigate. As with the gcc 3.2.3 compilation, my longobject.c patch hardly makes a difference. What does all this mean? Does the gcc 2.96 surprise tell us that small rearrangements in longobject.c could potentially boost performance by 45% on some platforms? Since I am not interested in optimizing longobject.c this is where I stopped. Eventually I decided that the problematic patch in longobject.c is not helping me with my original goal as stated near the beginning of this posting. I am only interested in counting the iterations of the interpreter loop. However, the _Py_Ticker decrements in longobject.c are not inside the interpreter loop, but in C loops! This means _Py_Ticker is useless for my purposes. Therefore I decoupled _Py_Ticker and _Py_TickerAccumulation. BTW: I think strictly speaking this documentation is incorrect because of the _Py_Ticker manipulations in longobject.c: sys.setcheckinterval.__doc__: Tell the Python interpreter to check for asynchronous events every n instructions. I feel very dizzy now after dealing with all the timings and the nasty details. A lot of fallout caused by the simple idea to add one innocent line to ceval.c. I hope it makes sense to somebody! Cheers, Ralf ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ diff -r -u original/Include/ceval.h work/Include/ceval.h --- original/Include/ceval.h 2004-10-10 19:40:35.000000000 -0700 +++ work/Include/ceval.h 2004-11-30 23:27:11.000000000 -0800 @@ -68,6 +68,11 @@ /* this used to be handled on a per-thread basis - now just two globals */ PyAPI_DATA(volatile int) _Py_Ticker; +#ifndef HAVE_LONG_LONG +PyAPI_DATA(volatile long) _Py_TickerAccumulation; +#else +PyAPI_DATA(volatile PY_LONG_LONG) _Py_TickerAccumulation; +#endif PyAPI_DATA(int) _Py_CheckInterval; /* Interface for threads. diff -r -u original/PC/os2emx/python24.def work/PC/os2emx/python24.def --- original/PC/os2emx/python24.def 2004-10-10 19:40:50.000000000 -0700 +++ work/PC/os2emx/python24.def 2004-11-30 23:27:11.000000000 -0800 @@ -743,6 +743,7 @@ "_Py_CheckRecursionLimit" "_Py_CheckInterval" "_Py_Ticker" + "_Py_TickerAccumulation" ; From python24_s.lib(compile) "PyCode_New" diff -r -u original/Python/ceval.c work/Python/ceval.c --- original/Python/ceval.c 2004-11-23 10:06:08.000000000 -0800 +++ work/Python/ceval.c 2004-12-03 19:39:36.000000000 -0800 @@ -373,6 +373,7 @@ pendinglast = j; _Py_Ticker = 0; + _Py_TickerAccumulation = 0; things_to_do = 1; /* Signal main loop */ busy = 0; /* XXX End critical section */ @@ -476,6 +477,11 @@ per thread, now just a pair o' globals */ int _Py_CheckInterval = 100; volatile int _Py_Ticker = 100; +#ifndef HAVE_LONG_LONG +volatile long _Py_TickerAccumulation = 0; +#else +volatile PY_LONG_LONG _Py_TickerAccumulation = 0; +#endif PyObject * PyEval_EvalCode(PyCodeObject *co, PyObject *globals, PyObject *locals) @@ -770,6 +776,7 @@ async I/O handler); see Py_AddPendingCall() and Py_MakePendingCalls() above. */ + _Py_TickerAccumulation++; if (--_Py_Ticker < 0) { if (*next_instr == SETUP_FINALLY) { /* Make the last opcode before diff -r -u original/Python/sysmodule.c work/Python/sysmodule.c --- original/Python/sysmodule.c 2004-08-12 11:19:17.000000000 -0700 +++ work/Python/sysmodule.c 2004-12-03 19:36:52.000000000 -0800 @@ -442,6 +442,20 @@ "getcheckinterval() -> current check interval; see setcheckinterval()." ); +static PyObject * +sys_gettickeraccumulation(PyObject *self, PyObject *args) +{ +#ifndef HAVE_LONG_LONG + return PyInt_FromLong(_Py_TickerAccumulation); +#else + return PyLong_FromLongLong(_Py_TickerAccumulation); +#endif +} + +PyDoc_STRVAR(gettickeraccumulation_doc, +"gettickeraccumulation() -> current count of bytecodes processed by the interpreter." +); + #ifdef WITH_TSC static PyObject * sys_settscdump(PyObject *self, PyObject *args) @@ -763,6 +777,8 @@ setcheckinterval_doc}, {"getcheckinterval", sys_getcheckinterval, METH_NOARGS, getcheckinterval_doc}, + {"gettickeraccumulation", sys_gettickeraccumulation, METH_NOARGS, + gettickeraccumulation_doc}, #ifdef HAVE_DLOPEN {"setdlopenflags", sys_setdlopenflags, METH_VARARGS, setdlopenflags_doc}, From bac at OCF.Berkeley.EDU Mon Dec 6 00:49:56 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Dec 6 00:51:03 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B34B62.7040201@v.loewis.de> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> Message-ID: <41B39EA4.9030105@ocf.berkeley.edu> Martin v. L?wis wrote: > Anthony Baxter wrote: > >> The library docs for, e.g. xmllib already say deprecated at the top - >> maybe >> it should be larger? > > > I think it would be better to *remove* the xmllib documentation. > Existing code which needs the module will continue to work even > without documentation, and new code is unlikely to be written for > a module that has no documentation, and where importing the module > gives a DeprecationWarning. > I like this solution. And if anyone cares about having good docs still for a deprecated module they can just read the docstrings. We don't get new people using it but others don't have to change their code. And as for the mention of dropping PEP 4, I agree with the running consensus that it isn't needed thanks to the warning module. Do we need to have a more formal vote to drop PEP 4, or should one the PEP maintainers just delete it? -Brett From bac at OCF.Berkeley.EDU Mon Dec 6 00:55:54 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Dec 6 00:57:05 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B2C991.4090502@v.loewis.de> References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de> Message-ID: <41B3A00A.30500@ocf.berkeley.edu> Martin v. L?wis wrote: > Brett C. wrote: > >> I noticed that Makefile.pre.in uses the value from the environment >> variable LDFLAGS but not CPPFLAGS. Any reason for this? > > > How did you notice that? For LDFLAGS, Makefile.pre.in has > > LDFLAGS= @LDFLAGS@ > > This does *not* mean that the value from the environment is used. > Instead, it means that configure computes the value of LDFLAGS > when it generates Makefile.in. For CPPFLAGS, configure has nothing > to compute, so Makefile.pre.in just has the static value for > CPPFLAGS. > I realize that much. But if you look in configure.in it seems to use the previous value of LDFLAGS every time it is redefined as a base and thus gets its initial value from the environment variable the first time it is tweaked. Not a big deal, though. I will just use the environment variables in setup.py . -Brett From python at rcn.com Mon Dec 6 01:05:56 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Dec 6 01:09:05 2004 Subject: [Python-Dev] proposal+patch: sys.gettickeraccumulation() In-Reply-To: <200412052337.iB5NbC5Y318887@boa.lbl.gov> Message-ID: <000601c4db27$6b1bad80$e841fea9@oemcomputer> > To restate my original goal: > > I am looking for a simple way to answer the question: How much of a > speedup can I expect if I reimplement a piece of Python code in C or > C++? . . . > Ratios (rounded to 3 decimals): > 16.9193/16.8831=1.002 > 5.8287/5.9553 =0.979 > 1.5944/1.5914 =1.002 > 10.7931/10.8149=0.998 > 5.2865/5.2528 =1.006 > 11.6086/11.6437=0.997 > 10.0992/11.0248=0.916 > 27.6830/27.6960=1.000 > > Therefore I'd argue that the runtime penalty for the one additional > long long increment in ceval.c (_Py_TickerAccumulation++) is in the > noise. The measurements are too imprecise to draw any worthwhile conclusions. Try running: python timeit.py -r9 "pass" That ought to give more stable measurements. The proposed "analysis tool" has no benefit to a majority of Python users. Even a 1% hit is not worth it. > I am only interested in counting the > iterations of the interpreter loop. However, the _Py_Ticker decrements > in longobject.c are not inside the interpreter loop, but in C loops! > This means _Py_Ticker is useless for my purposes. Therefore I decoupled > _Py_Ticker and _Py_TickerAccumulation. Why add this to everyone's build? Just put it in when doing your own analysis. The eval loop already pays a penalty for Py2.4's extra function tracing code. And ceval.c has been cluttered with #ifdefs for hardware timestamps. And there have been other additions for signal handling and whatnot. This is enough. >. A lot of fallout caused by the simple idea to add one innocent > line to ceval.c. I do not find it to be innocent. A great of work was expended over the years just trying to eliminate a small step or two from the eval-loop. Those efforts should not be discarded lightly. -1 on adding it directly. -0 on adding it as a #ifdeffed compile option (with the default being to exclude it). Raymond Hettinger From python at rcn.com Mon Dec 6 01:33:17 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Dec 6 01:36:01 2004 Subject: [Python-Dev] pystone rant [was] proposal+patch: sys.gettickeraccumulation() In-Reply-To: <200412052337.iB5NbC5Y318887@boa.lbl.gov> Message-ID: <000f01c4db2b$2e119f40$e841fea9@oemcomputer> (Old message) > > =) Parrotbench and PyBench would be enough for me to sign off on any > > performance hit. There is also pystone. pystone is good a predicting the relative performance of python apps on difference hardware/software environments. As a tool for evaluating proposed language changes, it completely sucks and should *never* be used for anything other than an extra informational datapoint. The foremost issue is that it exercises only a tiny portion of the language. Its design makes it worse than totally useless for evaluating eval-loop overhead. It runs a loop twice, once with benchmarking code and once without. Only the difference is reported. In theory, that means that all eval-loop speedups and slowdowns are netted out of the result. In practice, the reported result may indicate exactly the opposite of reality because the empty loop has vastly different cache effects than the benchmarked loop. For useful timings, run timeit on the specific feature in question. Then check for overall impact using pybench, parrotbench, and test_decimal. Raymond From barry at python.org Mon Dec 6 02:10:28 2004 From: barry at python.org (Barry Warsaw) Date: Mon Dec 6 02:10:31 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B39EA4.9030105@ocf.berkeley.edu> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> <41B39EA4.9030105@ocf.berkeley.edu> Message-ID: <1102295428.8361.8.camel@presto.wooz.org> On Sun, 2004-12-05 at 18:49, Brett C. wrote: > And as for the mention of dropping PEP 4, I agree with the running consensus > that it isn't needed thanks to the warning module. Do we need to have a more > formal vote to drop PEP 4, or should one the PEP maintainers just delete it? There's really no such thing as "dropping" a PEP, but in any event, we should still keep PEP 4 to document the procedure for deprecating modules. It just doesn't need to list any modules (i.e. remove the sections labeled Obsolete modules, Deprecated modules, and Undeprecated modules). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041205/c1ad06b4/attachment.pgp From anthony at interlink.com.au Mon Dec 6 02:19:31 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Dec 6 02:41:22 2004 Subject: [Python-Dev] pystone rant [was] proposal+patch: sys.gettickeraccumulation() In-Reply-To: <000f01c4db2b$2e119f40$e841fea9@oemcomputer> References: <000f01c4db2b$2e119f40$e841fea9@oemcomputer> Message-ID: <200412061219.32190.anthony@interlink.com.au> On Monday 06 December 2004 11:33, Raymond Hettinger wrote: > pystone is good a predicting the relative performance of python apps on > difference hardware/software environments. This comes up so often that I'm almost tempted to add a disclaimer to the pystone output. I can't count the number of times people have tried to claim that pystone showed that a change was good/neutral. From ncoghlan at iinet.net.au Mon Dec 6 09:53:42 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Dec 6 09:53:49 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B34B62.7040201@v.loewis.de> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> Message-ID: <41B41E16.90100@iinet.net.au> Martin v. L?wis wrote: > Anthony Baxter wrote: > >> The library docs for, e.g. xmllib already say deprecated at the top - >> maybe >> it should be larger? > > > I think it would be better to *remove* the xmllib documentation. > Existing code which needs the module will continue to work even > without documentation, and new code is unlikely to be written for > a module that has no documentation, and where importing the module > gives a DeprecationWarning. If we went down this path, I would suggest we include a list of deprecated modules in the docs to forestall "where is the documentation for these modules?" questions. Such a list would also make life easier for anyone stuck with maintaining legacy Python code. E.g. X.X Deprecated modules The following modules exist in the standard library solely for backwards compatibility with older versions of Python. They are no longer supported, and should not be used for new code. Anyone maintaining code using these modules should refer to the documentation for the Python version listed in the table below. Module Name | Last documented in: ------------------------------------- lib1 | Python 2.2 lib2 | Python 2.3 etc. . . I'd also suggest including a link to this list at the bottom of the global module index. >> Alternately, we could lock the folks complaining about the stdlib's state >> in a room with the folks who complain that every new thing has "wrecked >> the language that they knew and loved" and let them fight it out. > > That sounds like a fair and democratic way of solving the issue. If we sold tickets, it could even be a nice little earner for the PSF ;) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From mwh at python.net Mon Dec 6 13:12:48 2004 From: mwh at python.net (Michael Hudson) Date: Mon Dec 6 13:12:50 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B34B62.7040201@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sun, 05 Dec 2004 18:54:42 +0100") References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> Message-ID: <2mhdmzr5fj.fsf@starship.python.net> "Martin v. L?wis" <martin@v.loewis.de> writes: > Anthony Baxter wrote: >> The library docs for, e.g. xmllib already say deprecated at the top - maybe >> it should be larger? > > I think it would be better to *remove* the xmllib documentation. > Existing code which needs the module will continue to work even > without documentation, and new code is unlikely to be written for > a module that has no documentation, and where importing the module > gives a DeprecationWarning. +1, at least for *hiding* the documentation of xmllib + similar modules. I'm not sure wholesale removal is really a good idea. It's like the new "Non-essential Built-in Functions" section of the lib ref. Cheers, mwh -- <Acapnotic> jemfinch: What's to parse? A numeric code, perhaps a chicken, and some arguments -- from Twisted.Quotes From carribeiro at gmail.com Mon Dec 6 13:23:09 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Mon Dec 6 13:23:11 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <2mhdmzr5fj.fsf@starship.python.net> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> <2mhdmzr5fj.fsf@starship.python.net> Message-ID: <864d3709041206042352920501@mail.gmail.com> On Mon, 06 Dec 2004 12:12:48 +0000, Michael Hudson <mwh@python.net> wrote: > "Martin v. L?wis" <martin@v.loewis.de> writes: > > > Anthony Baxter wrote: > >> The library docs for, e.g. xmllib already say deprecated at the top - maybe > >> it should be larger? > > > > I think it would be better to *remove* the xmllib documentation. > > Existing code which needs the module will continue to work even > > without documentation, and new code is unlikely to be written for > > a module that has no documentation, and where importing the module > > gives a DeprecationWarning. > > +1, at least for *hiding* the documentation of xmllib + similar > modules. I'm not sure wholesale removal is really a good idea. 1) Move the deprecated documentation into a separate book. 2) Don't include the "Deprecated Modules Reference" in the standard distribution, but make it available as a separate download. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From ncoghlan at iinet.net.au Mon Dec 6 13:35:11 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Dec 6 13:35:18 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <864d3709041206042352920501@mail.gmail.com> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> <2mhdmzr5fj.fsf@starship.python.net> <864d3709041206042352920501@mail.gmail.com> Message-ID: <41B451FF.1000909@iinet.net.au> Carlos Ribeiro wrote: > On Mon, 06 Dec 2004 12:12:48 +0000, Michael Hudson <mwh@python.net> wrote: >>+1, at least for *hiding* the documentation of xmllib + similar >>modules. I'm not sure wholesale removal is really a good idea. > > > 1) Move the deprecated documentation into a separate book. > > 2) Don't include the "Deprecated Modules Reference" in the standard > distribution, but make it available as a separate download. What about the idea of pointing to the last version of the docs that contained the relevant documentation? Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From python at rcn.com Mon Dec 6 13:44:53 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Dec 6 13:47:37 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <864d3709041206042352920501@mail.gmail.com> Message-ID: <001101c4db91$627e4600$e841fea9@oemcomputer> > > > I think it would be better to *remove* the xmllib documentation. > > > Existing code which needs the module will continue to work even > > > without documentation, and new code is unlikely to be written for > > > a module that has no documentation, and where importing the module > > > gives a DeprecationWarning. > > > > +1, at least for *hiding* the documentation of xmllib + similar > > modules. I'm not sure wholesale removal is really a good idea. > 1) Move the deprecated documentation into a separate book. > > 2) Don't include the "Deprecated Modules Reference" in the standard > distribution, but make it available as a separate download. -1 We are trying to remove clutter, not keep it in perpetuity. Leaving the docs means continuing to have to test it, field bug reports, be aware of its existence, etc. Do not keep this on a respirator. Let it die in peace. Raymond From barry at python.org Mon Dec 6 13:55:35 2004 From: barry at python.org (Barry Warsaw) Date: Mon Dec 6 13:55:37 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <001101c4db91$627e4600$e841fea9@oemcomputer> References: <001101c4db91$627e4600$e841fea9@oemcomputer> Message-ID: <1102337735.8362.36.camel@presto.wooz.org> On Mon, 2004-12-06 at 07:44, Raymond Hettinger wrote: > We are trying to remove clutter, not keep it in perpetuity. Leaving the > docs means continuing to have to test it, field bug reports, be aware of > its existence, etc. > > Do not keep this on a respirator. Let it die in peace. Old documentation lives on in http://www.python.org/doc/versions.html which seems sufficient for looking up deprecated modules. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041206/5e733f08/attachment.pgp From bac at OCF.Berkeley.EDU Mon Dec 6 21:32:02 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Dec 6 21:32:24 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B2C991.4090502@v.loewis.de> References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de> Message-ID: <41B4C1C2.7060207@ocf.berkeley.edu> Martin v. L?wis wrote: > Brett C. wrote: > >> I noticed that Makefile.pre.in uses the value from the environment >> variable LDFLAGS but not CPPFLAGS. Any reason for this? > > > How did you notice that? For LDFLAGS, Makefile.pre.in has > > LDFLAGS= @LDFLAGS@ > > This does *not* mean that the value from the environment is used. > Instead, it means that configure computes the value of LDFLAGS > when it generates Makefile.in. For CPPFLAGS, configure has nothing > to compute, so Makefile.pre.in just has the static value for > CPPFLAGS. > I am not so sure that is true. Checking configure.in, there is no mention of CPPFLAGS anywhere. And yet if I modify the definition of CPPFLAGS in Makefile.pre.in to ``-I. -I./Include @CPPFLAGS@`` it ends up containing the value I have for the environment variable at the end of it. I think the '@@' syntax uses a value from configure.in if it is defined else it defaults to the value the shell has. -Brett From bac at OCF.Berkeley.EDU Mon Dec 6 07:36:14 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Dec 6 21:37:41 2004 Subject: [Python-Dev] Ptyon 2.3.5 probably coming in January; get your bugs/patches reported! Message-ID: <41B3FDDE.3040302@ocf.berkeley.edu> Anthony Baxter, our ever-diligent release manager, mentioned this past week that Python 2.3.5 will most likely come to fruition some time in January (this is not guaranteed date). This means that in order to have enough time to proper evaluate new patches and bugs they must be reported **now**! A one month lead time is necessary to properly look at, test, and commit patches, let alone coming up with solutions to any reported bugs. Please realize, though, that reporting a bug or submitting a patch now does not guarantee that it will committed in time! The free time of the development team is limited. If you want to help a bug or patch along to make it easier to be evaluated and thus raise its chances of being dealt with please see the "Helping Out" section of the 'Intro to Development' essay at http://www.python.org/dev/dev_intro.html . As always, both bugs and patches should be reported to Python's SourceForge tracker at http://sourceforge.net/bugs/?group_id=5470 and http://sourceforge.net/patch/?group_id=5470, respectively. -Brett Cannon From martin at v.loewis.de Mon Dec 6 22:28:39 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Dec 6 22:28:33 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <41B39EA4.9030105@ocf.berkeley.edu> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> <41B39EA4.9030105@ocf.berkeley.edu> Message-ID: <41B4CF07.4050108@v.loewis.de> Brett C. wrote: > And as for the mention of dropping PEP 4, I agree with the running > consensus that it isn't needed thanks to the warning module. Do we need > to have a more formal vote to drop PEP 4, or should one the PEP > maintainers just delete it? I would do what Barry suggests: rephrase the module to document the deprecation/removal procedure. This, of course, needs consensus/pronouncement, too, but I think I would put the following aspects into it: Requirements ============ Removal of module needs proper advance warning for users; the module should be removed only if - continued usage poses a security threat to the application using it, or - there are no reported usages of the module anymore for quite some time - the module is unmaintained (i.e. there are serious outstanding unfixed bugs for it), and there are alternatives which are maintained. There must be a way for users to stop removal of the module, e.g. by volunteering to maintain an unmaintained module. Deprecation =========== If a module is candidate for removal, it must be deprecated first. In order to deprecate the module 1. a warning must be added to the module, indicating the expected version when the module is removed, and a reference to the PEP 2. a comment must be added just above the warning, giving a the date and version of deprecation, and a rationale for removal (e.g.: no known users, is security risk) 3. the module documentation must be removed from the Python documentation (alternatively: moved into a "deprecated" section) Undeprecation ============= A module can be undeprecated if the deprecation reasons turns out to be invalid, e.g. if users of the module appear when the module was believed to have no users, or if a maintainer shows up for a previously unmaintained module. This fact must be recorded in the module; the original state (documentation, no warning) be restored. Removal ======= If the module has been deprecated for atleast a year and atleast a version, it can be removed. Removal should move it to old-libs for pure Python modules; a removal procedure for pure C modules has not been defined yet. If the module was deprecated because it was unmaintained, the module should only be removed if there have been no recent reports about usage of the module. Bug reports against deprecated modules ====================================== If a bug is reported against a deprecated module, the bug report can be closed without further consideration, pointing to the deprecation status. If a patch is submitted against the module, it should be considered whether the patch could undeprecate the module; if not, it can be rejected with the same rationale. If this policy can be agreed, I will replace PEP4 with it. Modules that have currently deprecation messages in them often fail to identify the Python version in which removal will occur. For modules that have been deprecated since 2.1, I would suggest to remove them for 2.5, with the option of bringing them back in 2.5.1 if people complain. Regards, Martin From martin at v.loewis.de Mon Dec 6 22:31:30 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Dec 6 22:31:23 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B3A00A.30500@ocf.berkeley.edu> References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de> <41B3A00A.30500@ocf.berkeley.edu> Message-ID: <41B4CFB2.6080907@v.loewis.de> Brett C. wrote: > I realize that much. But if you look in configure.in it seems to use > the previous value of LDFLAGS every time it is redefined as a base and > thus gets its initial value from the environment variable the first time > it is tweaked. Yes, it would be possible to do the same thing for CPPFLAGS. However, you really need to combine the values. If you assume you don't know anything about the current value of CPPFLAGS, this might get tricky - you might cause build failures if you honor CPPFLAGS too much. > Not a big deal, though. I will just use the environment variables in > setup.py . You shouldn't do this - you do need to consider the values in the Makefile as well. If you think you need both, you should modify configure.in appropriately. Regards, Martin From martin at v.loewis.de Mon Dec 6 22:39:30 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Dec 6 22:39:25 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B4C1C2.7060207@ocf.berkeley.edu> References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de> <41B4C1C2.7060207@ocf.berkeley.edu> Message-ID: <41B4D192.1040307@v.loewis.de> Brett C. wrote: >> How did you notice that? For LDFLAGS, Makefile.pre.in has >> >> LDFLAGS= @LDFLAGS@ >> >> This does *not* mean that the value from the environment is used. >> Instead, it means that configure computes the value of LDFLAGS >> when it generates Makefile.in. For CPPFLAGS, configure has nothing >> to compute, so Makefile.pre.in just has the static value for >> CPPFLAGS. >> > > I am not so sure that is true. Checking configure.in, there is no > mention of CPPFLAGS anywhere. Right. That's what I meant when I said "has nothing to compute", so it does not even mention CPPFLAGS. > And yet if I modify the definition of > CPPFLAGS in Makefile.pre.in to ``-I. -I./Include @CPPFLAGS@`` it ends up > containing the value I have for the environment variable at the end of > it. I think the '@@' syntax uses a value from configure.in if it is > defined else it defaults to the value the shell has. Indeed, that seems to be the case. However, absence of @CPPFLAGS@ means that Makefile.pre will just use the static value from Makefile.pre.in. Whether or not adding @CPPFLAGS@ to the end is the right thing, I don't know. Regards, Martin From barry at python.org Mon Dec 6 22:54:52 2004 From: barry at python.org (Barry Warsaw) Date: Mon Dec 6 22:55:04 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <41B4CF07.4050108@v.loewis.de> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> <41B39EA4.9030105@ocf.berkeley.edu> <41B4CF07.4050108@v.loewis.de> Message-ID: <1102370092.9565.30.camel@geddy.wooz.org> On Mon, 2004-12-06 at 16:28, "Martin v. L?wis" wrote: Martin, +1 on everything you wrote, with one minor quibble. > Removal > ======= > If the module has been deprecated for atleast a year and atleast > a version, it can be removed. Removal should move it to old-libs > for pure Python modules; a removal procedure for pure C modules > has not been defined yet. I wonder if the one year/one version rule is too short. Given that new versions come about every 18 months, I'd probably give it a 2 year/one version limit. > Modules that have currently deprecation messages in them often > fail to identify the Python version in which removal will occur. > For modules that have been deprecated since 2.1, I would suggest > to remove them for 2.5, with the option of bringing them back > in 2.5.1 if people complain +1 here too. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041206/54057bcd/attachment.pgp From eirik.mikkelsen at unix.net Mon Dec 6 23:58:40 2004 From: eirik.mikkelsen at unix.net (Eirik Mikkelsen) Date: Mon Dec 6 23:58:42 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B34B62.7040201@v.loewis.de> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> Message-ID: <1102373920.29451.4.camel@localhost> On Sun, 2004-12-05 at 18:54 +0100, "Martin v. L?wis" wrote: > I think it would be better to *remove* the xmllib documentation. > Existing code which needs the module will continue to work even > without documentation, and new code is unlikely to be written for > a module that has no documentation, and where importing the module > gives a DeprecationWarning. Not sure I'm a registered voter, but I'll chip in with a +1 If someone needs the documentation for xmllib it will still be available in the older versions. Eirik. From janssen at parc.com Tue Dec 7 01:25:07 2004 From: janssen at parc.com (Bill Janssen) Date: Tue Dec 7 01:25:23 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: Your message of "Sun, 05 Dec 2004 02:40:16 PST." <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> Message-ID: <04Dec6.162511pst."58617"@synergy1.parc.xerox.com> > * The average quality of the library improves as we take out junk (the > tzparse module for example) and put in high quality modules like > logging, csv, decimal, etc. Yes and no. The added modules have to be relevant to what users want to do. While (relatively) minor stuff like csv and decimal are good ideas, of course, logging is kind of an "insider's" module. What many more users want, however, are things like an XML parser, or a CSS parser, or a usable HTTP server, or... The fact that XML 1.1 can't be parsed with already-written Python is a *big* strike against. So removing highly demanded functionality like XML parsing, and adding specialist modules such as csv, do not overall help what users see as the quality of the standard library. Bill From t-meyer at ihug.co.nz Tue Dec 7 01:35:16 2004 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Tue Dec 7 01:36:03 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E8019B0136@its-xchg4.massey.ac.nz> Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C3854@its-xchg4.massey.ac.nz> >> * The average quality of the library improves as we take >> out junk (the tzparse module for example) and put in high >> quality modules like logging, csv, decimal, etc. > > Yes and no. The added modules have to be relevant to what > users want to do. While (relatively) minor stuff like csv > and decimal are good ideas, of course, logging is kind of an > "insider's" module. What do you mean by "insiders"? The logging module is great (ok, it could be simpler to use in some cases) for any Python programmer. > What many more users want, however, are things like an XML > parser, or a CSS parser, or a usable HTTP server, or... Statements like this are pretty common, but there's no evidence (that I've ever seen pointed to) that someone has *measured* how many people want modules for X. People who work with HTML at lot are probably keen on those things you list, yes. OTOH, other people (e.g. me) have no use for any of those, but use CSV and logging daily. Others want something completely different. There's quite a difference between quality and relevance. It's certainly worthwhile to ensure that all the standard library modules are as of high a quality as possible (e.g. email > rfc822). You'll never be able to get everyone to agree on the same set of modules that are relevant. If there are that many people that want (e.g.) a CSS parser, wouldn't there be a 3rd party one that everyone is using that could be proposed for addition into the standard library? =Tony.Meyer From python at rcn.com Tue Dec 7 01:43:07 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Dec 7 01:46:02 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <41B4CF07.4050108@v.loewis.de> Message-ID: <002401c4dbf5$b847cc00$f301a044@oemcomputer> > I would do what Barry suggests: rephrase the module to document the > deprecation/removal procedure. This, of course, needs > consensus/pronouncement, too, but I think I would put the following > aspects into it: > > Requirements > ============ > Removal of module needs proper advance warning for users; the > module should be removed only if > - continued usage poses a security threat to the application > using it, or > - there are no reported usages of the module anymore for quite > some time > - the module is unmaintained (i.e. there are serious outstanding > unfixed bugs for it), and there are alternatives which are > maintained. > There must be a way for users to stop removal of the module, > e.g. by volunteering to maintain an unmaintained module. The effect of coding this into the PEP is to make deprecation unnecessarily involved. Also, the list of reasons is insufficiently comprehensive. In the past we've gotten along fine with discussing a situation on python-dev and either reaching a consensus or having Guido give an okay. > 2. a comment must be added just above the warning, giving a > the date and version of deprecation, and a > rationale for removal (e.g.: no known users, is security risk) We should only talk in terms of versions. Dates are irrelevant and confusing. In the last version of the PEP, it was far from clear the significane of deprecating or removing on date xx-xx-xx. All that matters is the version the warning first appears and the version where the module is removed. > Modules that have currently deprecation messages in them often > fail to identify the Python version in which removal will occur. > For modules that have been deprecated since 2.1, I would suggest > to remove them for 2.5, with the option of bringing them back > in 2.5.1 if people complain. I do not think the version number messages are useful. When someone imports a file and gets a warning, they know a deprecation is pending and to stop writing code with that module. Further, I want to avoid the previous PEP 4 situation where one nit or another wasn't followed to the letter so we had to keep the module around for another two versions. It can and should be as simple as having us agree to deprecation, inserting a warning, and then removing it two versions later. Also, we need to shy away from the idea of having 2.5.1 with different capabilities than 2.5. This sort of thing is a portability disaster. Anthony, please speak up. General comments: I had thought the purpose of rewording the PEP was just to take out language naming specific modules and to add language on using the warning module. Instead, it looks like an attempt to make it much more difficult and slow to deprecate a module. If so, I would like to understand what situation is motivating it. Is there some module that had been deprecated but should not have been? Why the change in procedure? It doesn't affect me personally (I haven't proposed any modules for deprecation and do not have any in mind); however, I think the discussion should take place in the broader context of what we intend to do with the standard library -- do we want regex still there in 2010. Also, as proposed, the PEP lists many forces against deprecation and none of the forces for it. If that is the reality, then we could just never deprecate anything because someone, somewhere is bound to be irritated by it. That would certainly simplify the discussion. Going back to specifics, it may be worthwhile to think through the reason we kept the xmllib code but not whrandom. Both were documented, non-buggy, tested, marked as deprecated for a long time, and it was within the realm of possibility that some code still used them. Also, the PEP should discuss the use of the lib-old directory and it should include Barry's link to old documentation. Raymond From bac at OCF.Berkeley.EDU Tue Dec 7 01:52:08 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Dec 7 01:52:15 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B4D192.1040307@v.loewis.de> References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de> <41B4C1C2.7060207@ocf.berkeley.edu> <41B4D192.1040307@v.loewis.de> Message-ID: <41B4FEB8.90906@ocf.berkeley.edu> Martin v. L?wis wrote: > Brett C. wrote: > >>> How did you notice that? For LDFLAGS, Makefile.pre.in has >>> >>> LDFLAGS= @LDFLAGS@ >>> >>> This does *not* mean that the value from the environment is used. >>> Instead, it means that configure computes the value of LDFLAGS >>> when it generates Makefile.in. For CPPFLAGS, configure has nothing >>> to compute, so Makefile.pre.in just has the static value for >>> CPPFLAGS. >>> >> >> I am not so sure that is true. Checking configure.in, there is no >> mention of CPPFLAGS anywhere. > > > Right. That's what I meant when I said "has nothing to compute", so > it does not even mention CPPFLAGS. > >> And yet if I modify the definition of CPPFLAGS in Makefile.pre.in to >> ``-I. -I./Include @CPPFLAGS@`` it ends up containing the value I have >> for the environment variable at the end of it. I think the '@@' >> syntax uses a value from configure.in if it is defined else it >> defaults to the value the shell has. > > > Indeed, that seems to be the case. However, absence of @CPPFLAGS@ > means that Makefile.pre will just use the static value from > Makefile.pre.in. > That's basically the functionality I need, so I am going with it. > Whether or not adding @CPPFLAGS@ to the end is the right thing, > I don't know. > Well, we will soon find out. =) My checkin made this change and everything seems fine. If it doesn't work out I will have to create another environment variable to store the value. Michael's desire of getting the Fink and DarwinPorts special casing in setup.py is now solved; setup.py now uses the directories specified in LDFLAGS and CPPFLAGS for compiling the extension modules. I didn't bother with CFLAGS or CC since the former is mostly handled by BASECFLAGS it seems and the latter is specified by arguments to configure. -Brett From python at rcn.com Tue Dec 7 01:53:05 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Dec 7 01:55:58 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <002401c4dbf5$b847cc00$f301a044@oemcomputer> Message-ID: <002801c4dbf7$1ce6a860$f301a044@oemcomputer> One other thought: In deciding how long to leave module in, we should consider that Python books are updated infrequently, if at all. It would be a bummer if code in them stopped working as advertised. Raymond From anthony at interlink.com.au Tue Dec 7 02:31:13 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Dec 7 02:31:29 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <002401c4dbf5$b847cc00$f301a044@oemcomputer> References: <002401c4dbf5$b847cc00$f301a044@oemcomputer> Message-ID: <200412071231.13878.anthony@interlink.com.au> On Tuesday 07 December 2004 11:43, Raymond Hettinger wrote: > > Modules that have currently deprecation messages in them often > > fail to identify the Python version in which removal will occur. > > For modules that have been deprecated since 2.1, I would suggest > > to remove them for 2.5, with the option of bringing them back > > in 2.5.1 if people complain. > > [...] > > Also, we need to shy away from the idea of having 2.5.1 with different > capabilities than 2.5. This sort of thing is a portability disaster. > Anthony, please speak up. Sorry - been a bit busy the last few days. Raymond is right here - there's no way we can or should do this. A bugfix release contains _bugfixes_. Making new modules available in a bugfix is a sucky sucky approach. Worse - once we remove a module, it should _stay_ removed. Otherwise, we're going to end up with crap like: if sys.version < (2,5) or sys.version >= (2,6): import froggie else: # froggie was removed in 2.5 and reinstated in 2.6 from compat import froggie and people will be shipping their own versions of the code to get around our misfeature. -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood. From janssen at parc.com Tue Dec 7 02:49:40 2004 From: janssen at parc.com (Bill Janssen) Date: Tue Dec 7 02:50:15 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: Your message of "Mon, 06 Dec 2004 16:35:16 PST." <ECBA357DDED63B4995F5C1F5CBE5B1E86C3854@its-xchg4.massey.ac.nz> Message-ID: <04Dec6.174948pst."58617"@synergy1.parc.xerox.com> > Statements like this are pretty common, but there's no evidence (that I've > ever seen pointed to) that someone has *measured* how many people want > modules for X. I almost didn't send this in, because I figured someone would have to argue with it. > If there are that many people that want (e.g.) a CSS parser, wouldn't there > be a 3rd party one that everyone is using that could be proposed for > addition into the standard library? As far as I can tell, there are no CSS or XML 1.1 parsers for Python, period. Bill From t-meyer at ihug.co.nz Tue Dec 7 03:04:49 2004 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Tue Dec 7 03:05:23 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <ECBA357DDED63B4995F5C1F5CBE5B1E8019B0164@its-xchg4.massey.ac.nz> Message-ID: <ECBA357DDED63B4995F5C1F5CBE5B1E86C3858@its-xchg4.massey.ac.nz> > As far as I can tell, there are no CSS or XML 1.1 parsers for > Python, period. This belongs on c.l.p, I suppose, but the first page of google results includes: <http://www.python.org/pypi?:action=display&name=TG%20CSS%20Tools&version=1. 0a1> <http://cthedot.de/cssutils/> =Tony.Meyer From sjoerd at acm.org Mon Dec 6 21:47:37 2004 From: sjoerd at acm.org (Sjoerd Mullender) Date: Tue Dec 7 03:19:07 2004 Subject: [Python-Dev] Any reason why CPPFLAGS not used in compiling? In-Reply-To: <41B4C1C2.7060207@ocf.berkeley.edu> References: <41B2BE2E.3000907@ocf.berkeley.edu> <41B2C991.4090502@v.loewis.de> <41B4C1C2.7060207@ocf.berkeley.edu> Message-ID: <41B4C569.8060608@acm.org> Brett C. wrote: > Martin v. L?wis wrote: > >> Brett C. wrote: >> >>> I noticed that Makefile.pre.in uses the value from the environment >>> variable LDFLAGS but not CPPFLAGS. Any reason for this? >> >> >> >> How did you notice that? For LDFLAGS, Makefile.pre.in has >> >> LDFLAGS= @LDFLAGS@ >> >> This does *not* mean that the value from the environment is used. >> Instead, it means that configure computes the value of LDFLAGS >> when it generates Makefile.in. For CPPFLAGS, configure has nothing >> to compute, so Makefile.pre.in just has the static value for >> CPPFLAGS. >> > > I am not so sure that is true. Checking configure.in, there is no > mention of CPPFLAGS anywhere. And yet if I modify the definition of > CPPFLAGS in Makefile.pre.in to ``-I. -I./Include @CPPFLAGS@`` it ends up > containing the value I have for the environment variable at the end of > it. I think the '@@' syntax uses a value from configure.in if it is > defined else it defaults to the value the shell has. It's autoconf that deals with these flags. See the output of "configure --help". -- Sjoerd Mullender <sjoerd@acm.org> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 374 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20041206/bfe3b473/signature.pgp From kbk at shore.net Tue Dec 7 05:33:45 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Dec 7 05:35:36 2004 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200412070433.iB74XjGf019407@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 259 open ( +1) / 2705 closed ( +4) / 2964 total ( +5) Bugs : 800 open (-12) / 4662 closed (+20) / 5462 total ( +8) RFE : 160 open ( +0) / 137 closed ( +1) / 297 total ( +1) New / Reopened Patches ______________________ add key (2004-12-01) CLOSED http://python.org/sf/1077353 opened by Steven Bethard Simple webbrowser fix for netscape -remote (2004-12-02) http://python.org/sf/1077979 opened by Josh Cherry readline does not need termcap (2004-12-01) http://python.org/sf/1076826 reopened by nijel Make cgi.py use logging module (2004-12-05) http://python.org/sf/1079729 opened by Josh Hoyt Make cgi.py use email instead of rfc822 or mimetools (2004-12-05) http://python.org/sf/1079734 opened by Josh Hoyt list sort is not "in place" (2004-12-06) http://python.org/sf/1080078 reopened by rhettinger list sort is not "in place" (2004-12-06) CLOSED http://python.org/sf/1080078 opened by Heikki Orsila Patches Closed ______________ add key (2004-12-02) http://python.org/sf/1077353 closed by rhettinger readline does not need termcap (2004-12-01) http://python.org/sf/1076826 closed by loewis raise error for common mistake with subprocess (2004-11-23) http://python.org/sf/1071755 closed by astrand list sort is not "in place" (2004-12-06) http://python.org/sf/1080078 closed by rhettinger list sort is not "in place" (2004-12-06) http://python.org/sf/1080078 closed by tim_one AUTH PLAIN in smtplib (2004-11-30) http://python.org/sf/1075928 closed by jlgijsbers New / Reopened Bugs ___________________ problem with make test on OS/X (2004-12-02) CLOSED http://python.org/sf/1077302 opened by Ian Holsman threads: segfault or Py_FatalError at exit (2004-11-08) http://python.org/sf/1061968 reopened by mhammond assert fail to threw exception when run python with '-O' (2004-12-02) CLOSED http://python.org/sf/1077862 opened by tj128 Python2.4: building '_socket' extension fails with `INET_ADD (2004-12-03) http://python.org/sf/1078245 opened by Michael Str?der Docs for run() methods are misleading (2004-12-04) CLOSED http://python.org/sf/1078905 opened by Kent Johnson Email.Header encodes non-ASCII content incorrectly (2004-12-04) http://python.org/sf/1078919 opened by Tessa Lau Incorrect error message (somewhat) (2004-12-04) http://python.org/sf/1079011 opened by Gerrit Holl datetime changes missing from "Porting from 2.3 to 2.4" (2004-12-04) http://python.org/sf/1079134 opened by Sadruddin Rejeb python-2.4.msi install error (2004-12-05) http://python.org/sf/1079545 opened by maharal IDLE and old color themes (2004-12-06) http://python.org/sf/1080387 opened by projecktzero Bugs Closed ___________ threads: segfault or Py_FatalError at exit (2004-11-07) http://python.org/sf/1061968 closed by arigo Tutorial corrections (2004-12-01) http://python.org/sf/1076955 closed by rhettinger test_shutil fails on x86-64 // Suse 9.1 (2004-12-01) http://python.org/sf/1076467 closed by jlgijsbers test test_unicodedata failed (2004-12-01) http://python.org/sf/1076795 closed by lemburg Problem testing python 2.4 (2004-12-01) http://python.org/sf/1077103 closed by lemburg re module segfaulting in large regular expression (2004-11-24) http://python.org/sf/1072259 closed by niemeyer Memory fault pyexpat.so on SGI (2004-11-30) http://python.org/sf/1075990 closed by akuchling assert fail to threw exception when run python with '-O' (2004-12-02) http://python.org/sf/1077862 closed by tim_one Errors and omissions in logging module documentation (2004-11-28) http://python.org/sf/1074693 closed by vsajip xml.dom.minidom produces errors with certain unicode chars (2004-11-27) http://python.org/sf/1074200 closed by effbot test test_re produced unexpected output (2004-12-01) http://python.org/sf/1076791 closed by nijel ^Z doesn't exit interpreter - 2.4c1 & Win2K (2004-11-26) http://python.org/sf/1073736 closed by facundobatista test_macostools failure on OS X (2004-12-01) http://python.org/sf/1077302 closed by bcannon PyGILState_Ensure() deadlocks (ver 2.4) (2004-11-29) http://python.org/sf/1075703 closed by andivajda calendar.weekheader() undocumented (2004-05-04) http://python.org/sf/947894 closed by doerwalter Docs for unittest run() methods are misleading (2004-12-04) http://python.org/sf/1078905 closed by rhettinger Another message that croaks email.FeedParser (2004-11-30) http://python.org/sf/1076485 closed by bwarsaw Windows msi doesn't install site-packages directory (2004-11-23) http://python.org/sf/1071594 closed by loewis win32con missing codes VK_VOLUME_MUTE, VK_BROWSER_BACK, ... (2004-11-16) http://python.org/sf/1067153 closed by loewis test_sutil fails on cygwin (2004-11-23) http://python.org/sf/1071513 closed by jlgijsbers CFLAGS, CPPFLAGS, LDFLAGS are NOT used when building modules (2004-05-17) http://python.org/sf/955197 closed by bcannon RFE Closed __________ Python Interpreter embedded in small memory system (2004-11-30) http://python.org/sf/1076478 closed by rhettinger From martin at v.loewis.de Tue Dec 7 08:08:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Dec 7 08:08:39 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <002401c4dbf5$b847cc00$f301a044@oemcomputer> References: <002401c4dbf5$b847cc00$f301a044@oemcomputer> Message-ID: <41B556FE.20504@v.loewis.de> Raymond Hettinger wrote: > The effect of coding this into the PEP is to make deprecation > unnecessarily involved. Can you please explain why you consider this involvement unnecessary? I think it is important that we do not prematurely remove (or even deprecate) modules that are still being actively used. > Also, the list of reasons is insufficiently > comprehensive. In the past we've gotten along fine with discussing a > situation on python-dev and either reaching a consensus or having Guido > give an okay. I don't think we "got along", but rather "got away". Discussion on python-dev does not cause sufficient user involvement, IMO. And indeed, some people (even on python-dev) think that we *never* should deprecate any modules, as it breaks existing code. They might not be as vocal as supporters of early removal of "ugly" code, but I think their opinion is just as valuable. > We should only talk in terms of versions. Dates are irrelevant and > confusing. Why is a date confusing? I think everybody on Earth understands the notion of a date, there is nothing confusing in this notion. Why are they irrelevant? If we release Python 2.5 next month (and stop developing 2.4), are we entitled to remove all features that were scheduled for removal? I think not: users would have not gotten sufficient advance warning. Users need *time* to react on proposed feature changes. > In the last version of the PEP, it was far from clear the > significane of deprecating or removing on date xx-xx-xx. All that > matters is the version the warning first appears and the version where > the module is removed. No. It also matters how much time users have had to react. > I do not think the version number messages are useful. When someone > imports a file and gets a warning, they know a deprecation is pending > and to stop writing code with that module. Yes, but what about code that currently uses the module? People will not start rewriting (perhaps they aren't the authors in the first place) just because they see a deprecation warning. Instead, they will silence the warning, and go on. If we plan to remove the feature at some point, we need to give these users an indication that they need to act beyond silencing the warning. Such actions could be - have the author of the software update it appropriately - look for a replacement software - ask for an extension of the grace period on python-dev Currently, users do not really have these options, since they do not have an indication that action on their side is really needed if they want their software continue to work with future Python versions. > Further, I want to avoid the > previous PEP 4 situation where one nit or another wasn't followed to the > letter so we had to keep the module around for another two versions. Why do you want to avoid that situation? What is the problem with waiting for two more versions? No harm is done in doing so. > It > can and should be as simple as having us agree to deprecation, inserting > a warning, and then removing it two versions later. I strongly disagree with that position. > I had thought the purpose of rewording the PEP was just to take out > language naming specific modules and to add language on using the > warning module. Instead, it looks like an attempt to make it much more > difficult and slow to deprecate a module. I want the PEP to be followed. For that, it needs to be precise, and the procedures need to be agreed upon (or, if we cannot agree, it needs to be Pronounced). > If so, I would like to understand what situation is motivating it. Is > there some module that had been deprecated but should not have been? Deprecation is fine for all the modules so far. Generating the warning is a bit of a problem, removal would be a serious problem, for the following modules: - regex - xmllib - whrandom since I expect that these modules are still widely used, in existing code, by users who are not the authors of that code. This is a problem, since these users cannot do anything about the usage of the deprecated code. They just find that the software that ran happily with the previous Python version first shouts at them in the next Python version, then totally breaks with the subsequent version. They will hate Python for that. > Why the change in procedure? I think we agree that the previously-documented procedure needs to be changed - it was not followed. > It doesn't affect me personally (I haven't proposed any modules for > deprecation and do not have any in mind); however, I think the > discussion should take place in the broader context of what we intend to > do with the standard library -- do we want regex still there in 2010. As to the latter question: Definitely. We don't want regex to be used for new code, and we want developers to fully understand that, but we also do want code that currently works correctly with regex continue to do so in 2010. The only exception would be that 2010 no code is left in active use that uses regex, which I doubt. > Also, as proposed, the PEP lists many forces against deprecation and > none of the forces for it. That is not true. Security flaws are giving priority. They can only be undeprecated if the reason for deprecation goes away, which means that somebody fixes the security flaw. > If that is the reality, then we could just > never deprecate anything because someone, somewhere is bound to be > irritated by it. That would certainly simplify the discussion. Well, we do want to move developers to the new direction. However, many people use Python code who are *not* developers. We need to take their concerns into account as well. > Going back to specifics, it may be worthwhile to think through the > reason we kept the xmllib code but not whrandom. Both were documented, > non-buggy, tested, marked as deprecated for a long time, and it was > within the realm of possibility that some code still used them. Indeed. If it is in the realm of possibility that code still uses a module, and that this would not raise security concerns, the module should not be removed. > Also, the PEP should discuss the use of the lib-old directory and it > should include Barry's link to old documentation. I'm happy to adjust details of the procedures - but it seems we disagree on the principles. Regards, Martin From martin at v.loewis.de Tue Dec 7 08:12:21 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Dec 7 08:12:14 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <002801c4dbf7$1ce6a860$f301a044@oemcomputer> References: <002801c4dbf7$1ce6a860$f301a044@oemcomputer> Message-ID: <41B557D5.2080700@v.loewis.de> Raymond Hettinger wrote: > One other thought: In deciding how long to leave module in, we should > consider that Python books are updated infrequently, if at all. It > would be a bummer if code in them stopped working as advertised. Correct. Thanks for reminding me - that is indeed a reasoning that is typically brought up by book authors, so I should have brought it up myself :-) Of course, developers would first see the deprecation warning, which would tell them that their books are outdated. However, if the code is removed, they get an import error, which leaves them guessing what the problem might have been. Regards, Martin From python at rcn.com Tue Dec 7 08:50:55 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Dec 7 08:53:39 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <41B556FE.20504@v.loewis.de> Message-ID: <000701c4dc31$7bf6da20$e841fea9@oemcomputer> > I'm happy to adjust details of the procedures - but it seems we disagree > on the principles. Not really. I have no objection to making module deprecation/removal rare or even not doing it at all. Personally, I've never asked for a module deprecation and don't expect to. I also agree that more public participation is a wise step. I would just like to see a clean, actionable PEP. To me, the draft text outlined a timid, faltering process that would be hard to follow, prone to reversal, and accomplish little. I especially find burdensome the obligation to undo a deprecation the moment some random user sends a grumpy email. Instead, there should be a clear decision to deprecate or not. If that entails a comp.lang.python.announce notice and comment period, so be it. Once a decision is made, document it, add a warning, and wait. Once a couple versions pass, some useful action needs to take place. If the code is left in-place and the doc index is just re-arranged, then we would have been better off not doing anything at all. Ideally, the PEP should also provide some informational value. It should list Barry's link as a reference for old docs. It should describe the appropriate use of lib-old (like whrandom) vs. renaming a module (like stringold) vs. leaving it in-place (like xmllib) vs. removing the module The questions of dates was somewhat minor. I was unclear on the implication of, for example, "remove on 15Jan2005". Since Py2.5 won't go out for at least a year, does that mean that the work can't be done now while I have time instead of later when I don't. The only time the removal becomes visible is when a new version goes out the door. Further, if a version is going to have something removed, we should do it at the outset rather than just before a release goes out the door (to allow for early surfacing of any issues). > > Further, I want to avoid the > > previous PEP 4 situation where one nit or another wasn't followed to the > > letter so we had to keep the module around for another two versions. > > Why do you want to avoid that situation? What is the problem with > waiting for two more versions? No harm is done in doing so. If we really don't care whether it gets done, then we shouldn't bother in the first place. Raymond From python at rcn.com Tue Dec 7 09:12:20 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Dec 7 09:15:03 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <000701c4dc31$7bf6da20$e841fea9@oemcomputer> Message-ID: <000801c4dc34$7949b420$e841fea9@oemcomputer> There is one other issue that should get addressed, modules in limbo. I'm concerned about the old email modules. They haven't been deprecated and may stay around for a good while. However, some SF requests have been passed over on the basis that the modules are supplanted and will be phased out. I don't think it is good to leave active modules as orphans. Raymond From martin at v.loewis.de Tue Dec 7 09:16:05 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Dec 7 09:15:58 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <000701c4dc31$7bf6da20$e841fea9@oemcomputer> References: <000701c4dc31$7bf6da20$e841fea9@oemcomputer> Message-ID: <41B566C5.2040102@v.loewis.de> Raymond Hettinger wrote: > Instead, there should be a clear decision to deprecate or not. If that > entails a comp.lang.python.announce notice and comment period, so be it. > Once a decision is made, document it, add a warning, and wait. Ok, that might be reasonable. > Once a couple versions pass, some useful action needs to take place. If > the code is left in-place and the doc index is just re-arranged, then we > would have been better off not doing anything at all. I disagree. The primary purpose (move developers to the alternative approach) is achieved as soon as the warning is added, and the documentation is deleted. Whether the module is actually deleted is of minor importance: it costs nothing to continue including it; disk space is cheap. > The questions of dates was somewhat minor. I was unclear on the > implication of, for example, "remove on 15Jan2005". Since Py2.5 won't > go out for at least a year, does that mean that the work can't be done > now while I have time instead of later when I don't. The only time the > removal becomes visible is when a new version goes out the door. You could remove it now, but if we release Py2.5 in two months, you would have to put it back in. So if you don't have time then, maybe somebody else will. If nobody has time to remove the module, the next release will ship with it, again - no big issue. > Further, if a version is going to have something removed, we should do > it at the outset rather than just before a release goes out the door (to > allow for early surfacing of any issues). That is true. > If we really don't care whether it gets done, then we shouldn't bother > in the first place. What do you mean by "bother"? Not put the deprecation warning in? We do want users to move to the new approach of doing something. However, two version are just not enough - it may take 10 or 20 years to remove a widely used feature (e.g. the string module). That it will take so long does not mean we should not start the process *now* - if we start the process in five years, it will *still* take 10 or 20 years. Just be patient. Regards, Martin From python at rcn.com Tue Dec 7 09:35:57 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Dec 7 09:38:42 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <41B566C5.2040102@v.loewis.de> Message-ID: <000e01c4dc37$c673b400$e841fea9@oemcomputer> > > Instead, there should be a clear decision to deprecate or not. If that > > entails a comp.lang.python.announce notice and comment period, so be it. > > Once a decision is made, document it, add a warning, and wait. > > Ok, that might be reasonable. Please word the PEP accordingly. > > Once a couple versions pass, some useful action needs to take place. If > > the code is left in-place and the doc index is just re-arranged, then we > > would have been better off not doing anything at all. > > I disagree. The primary purpose (move developers to the alternative > approach) is achieved as soon as the warning is added, and the > documentation is deleted. Whether the module is actually deleted is > of minor importance: it costs nothing to continue including it; disk > space is cheap. That is reasonable. Please make that goal/assumption explicit in the PEP. > > The questions of dates was somewhat minor. I was unclear on the > > implication of, for example, "remove on 15Jan2005". Since Py2.5 won't > > go out for at least a year, does that mean that the work can't be done > > now while I have time instead of later when I don't. The only time the > > removal becomes visible is when a new version goes out the door. > > You could remove it now, but if we release Py2.5 in two months, you > would have to put it back in. So if you don't have time then, maybe > somebody else will. If nobody has time to remove the module, the next > release will ship with it, again - no big issue. Okay. Again, please say that in the PEP. > > Further, if a version is going to have something removed, we should do > > it at the outset rather than just before a release goes out the door (to > > allow for early surfacing of any issues). > > That is true. That should also be recommended in the PEP. > > If we really don't care whether it gets done, then we shouldn't bother > > in the first place. > > What do you mean by "bother"? Not put the deprecation warning in? We > do want users to move to the new approach of doing something. However, > two version are just not enough - it may take 10 or 20 years to remove > a widely used feature (e.g. the string module). That it will take so > long does not mean we should not start the process *now* - if we start > the process in five years, it will *still* take 10 or 20 years. Just > be patient. I see. That also may useful to include in the motivation section of the PEP. With these adjustments, I think the PEP will be fine. Also, be sure get rid of the mid-version undo (see Anthony's note) and to address the situation with Python books. I look forward to a revised draft. Raymond From ncoghlan at iinet.net.au Tue Dec 7 09:39:06 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Dec 7 09:39:12 2004 Subject: [Python-Dev] Re: long number multiplication In-Reply-To: <t1wfz2jfn9p.fsf@kermit.wreck.org> References: <Pine.LNX.4.44.0412050900300.12327-100000@localhost> <t1wfz2jfn9p.fsf@kermit.wreck.org> Message-ID: <41B56C2A.3030609@iinet.net.au> Christopher A. Craig wrote: >>i needed to implement this myself and was thinking of storing the digits >>of an integer in a list. > > > That's sort of what Python does except the "digits" are 15 bits, > not base 10. Doing it in base 10 would be a huge pain because of the > problems with base 10->base 2 conversion. > Indeed. Python's Decimal data type stores the digits in a list for ease of power of 10 multiplication & rounding (the precision rules means a *lot* of that goes on). However, it converts that list of digits to a long integer in order to do addition, multiplication or division. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Tue Dec 7 10:46:41 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Dec 7 10:46:49 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <000e01c4dc37$c673b400$e841fea9@oemcomputer> References: <000e01c4dc37$c673b400$e841fea9@oemcomputer> Message-ID: <41B57C01.9050404@iinet.net.au> One other question occurred to me for "deprecate X in favour of Y" situations - should the deprecation warning added to X point developers towards Y? Also, should the PEP include example wordings for deprecation warnings, rather than being completely freeform? Martin listed some information that should be included, but it seems an example or two would make it easy to get right. E.g.: Unmaintained, with a maintained alternative: "Due to lack of maintenance, this module has been deprecated in favour of module Y and will be removed in Python 2.6 (see PEP 4 for information on the deprecation process)" Security problems, no alternative: "Due to security concerns, this module has been deprecated and will be removed in Python 2.6 (see PEP 4 for information on the deprecation process)" Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Tue Dec 7 10:51:01 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Dec 7 10:51:10 2004 Subject: [Python-Dev] Re: long number multiplication In-Reply-To: <41B56C2A.3030609@iinet.net.au> References: <Pine.LNX.4.44.0412050900300.12327-100000@localhost> <t1wfz2jfn9p.fsf@kermit.wreck.org> <41B56C2A.3030609@iinet.net.au> Message-ID: <41B57D05.1090509@iinet.net.au> Gah, sorry folks. I really ought to pay more attention to that send line. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From barry at python.org Tue Dec 7 15:06:10 2004 From: barry at python.org (Barry Warsaw) Date: Tue Dec 7 15:06:15 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <000801c4dc34$7949b420$e841fea9@oemcomputer> References: <000801c4dc34$7949b420$e841fea9@oemcomputer> Message-ID: <1102428369.25008.31.camel@geddy.wooz.org> On Tue, 2004-12-07 at 03:12, Raymond Hettinger wrote: > I'm concerned about the old email modules. They haven't been deprecated > and may stay around for a good while. However, some SF requests have > been passed over on the basis that the modules are supplanted and will > be phased out. I don't think it is good to leave active modules as > orphans. I would like to add deprecation warnings to some of the old email-related modules for Python 2.5, but I doubt we can remove any of them until Python 3.0 (or 2014, whichever comes first <wink>). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041207/2226a04a/attachment.pgp From fdrake at acm.org Tue Dec 7 15:47:37 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue Dec 7 15:48:02 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <41B57C01.9050404@iinet.net.au> References: <000e01c4dc37$c673b400$e841fea9@oemcomputer> <41B57C01.9050404@iinet.net.au> Message-ID: <200412070947.38173.fdrake@acm.org> On Tuesday 07 December 2004 04:46 am, Nick Coghlan wrote: > One other question occurred to me for "deprecate X in favour of Y" > situations - should the deprecation warning added to X point developers > towards Y? Not sure about the warning, but the documentation certainly should. When the mapping from the old module to the new is not obvious and trivial, sufficient information should be included in the documentation for the old module to make it fairly easy to perform the conversion. (Placing this in the docs for the old module ensures that it goes away when the documentation for the old module goes away, and the docs for the new module need never be loaded with information about the old module.) > Also, should the PEP include example wordings for deprecation warnings, > rather than being completely freeform? Martin listed some information that > should be included, but it seems an example or two would make it easy to > get right. Examples would be good, though specific wording should not be proscribed. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> From mwh at python.net Tue Dec 7 15:54:50 2004 From: mwh at python.net (Michael Hudson) Date: Tue Dec 7 15:54:53 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_shutil.py, 1.10, 1.11 In-Reply-To: <E1CbPoH-0005LW-Qo@sc8-pr-cvs1.sourceforge.net> (jlgijsbers@users.sourceforge.net's message of "Mon, 06 Dec 2004 12:50:17 -0800") References: <E1CbPoH-0005LW-Qo@sc8-pr-cvs1.sourceforge.net> Message-ID: <2m4qiyqhtx.fsf@starship.python.net> jlgijsbers@users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Lib/test > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20531 > > Modified Files: > test_shutil.py > Log Message: > SF bug #1076467: don't run test_on_error as root, as the permission > errors don't get provoked that way. Also add a bunch of cross-references > to bugs. > > > Index: test_shutil.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/test/test_shutil.py,v > retrieving revision 1.10 > retrieving revision 1.11 > diff -u -d -r1.10 -r1.11 > --- test_shutil.py 23 Nov 2004 09:27:27 -0000 1.10 > +++ test_shutil.py 6 Dec 2004 20:50:15 -0000 1.11 > @@ -16,7 +16,10 @@ > filename = tempfile.mktemp() > self.assertRaises(OSError, shutil.rmtree, filename) > > - if hasattr(os, 'chmod') and sys.platform[:6] != 'cygwin': > + # See bug #1071513 for why we don't run this on cygwin > + # and bug #1076467 for why we don't run this as root. > + if (hasattr(os, 'chmod') and sys.platform[:6] != 'cygwin' > + and os.getenv('USER') != 'root'): Is that really the best way to check for root? It may be, I guess, but I'd have expected os.geteuid() to be more reliable... Or is it os.getuid()? I always get confused by these functions... Cheers, mwh -- <glyph> CDATA is not an integration strategy. -- from Twisted.Quotes From lists at andreas-jung.com Tue Dec 7 18:24:31 2004 From: lists at andreas-jung.com (Andreas Jung) Date: Tue Dec 7 18:24:37 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL Message-ID: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]> While using Zope 2.7 with Python 2.4 we discovered some strange behaviour of the security machinery. I could track this down to some Zope code in cAccessControl.c where an Unauthorized exception is raised because of a call to PyInt_FromLong(1) which returns NULL. What could be the reason that such a "stupid" call return NULL in a reproducable way? -aj From mwh at python.net Tue Dec 7 18:28:45 2004 From: mwh at python.net (Michael Hudson) Date: Tue Dec 7 18:28:47 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]> (Andreas Jung's message of "Tue, 07 Dec 2004 18:24:31 +0100") References: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]> Message-ID: <2mzn0qow4y.fsf@starship.python.net> Andreas Jung <lists@andreas-jung.com> writes: > While using Zope 2.7 with Python 2.4 we discovered some strange > behaviour of the security machinery. > I could track this down to some Zope code in cAccessControl.c where an > Unauthorized exception is > raised because of a call to PyInt_FromLong(1) which returns NULL. What > could be the reason that > such a "stupid" call return NULL in a reproducable way? A memory scribble somewhere else? Possibly a DECREF too many somewhere? Is an exception set? Have you tried a debug build? Etc. Cheers, mwh -- All obscurity will buy you is time enough to contract venereal diseases. -- Tim Peters, python-dev From jim at zope.com Tue Dec 7 18:30:19 2004 From: jim at zope.com (Jim Fulton) Date: Tue Dec 7 18:30:24 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]> References: <9DFB75B2DC037BC6AAD05F9E@[192.168.0.102]> Message-ID: <41B5E8AB.8060405@zope.com> Andreas Jung wrote: > While using Zope 2.7 with Python 2.4 we discovered some strange > behaviour of the security machinery. > I could track this down to some Zope code in cAccessControl.c where an > Unauthorized exception is > raised because of a call to PyInt_FromLong(1) which returns NULL. What > could be the reason that > such a "stupid" call return NULL in a reproducable way? Ugh. Part of the problem is that all of those calls are unchecked, Dang us. If they were checked, then, who knows, we might have gotten informative exceptions. I'd say the first step should be to add checks. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From tim.peters at gmail.com Tue Dec 7 18:58:12 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 7 18:59:03 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> Message-ID: <1f7befae041207095831b00ae0@mail.gmail.com> [Andreas Jung] > While using Zope 2.7 Do you mean 2.7, or do you mean 2.7.3, or ...? > with Python 2.4 we discovered some strange behaviour > of the security machinery. > I could track this down to some Zope code in cAccessControl.c > where an Unauthorized exception is raised because of a call to > PyInt_FromLong(1) which returns NULL. What could be the > reason that such a "stupid" call return NULL in a reproducable > way? Any C function that returns a Python object can return NULL if malloc() says there's not enough memory to create a new object. PyInt_FromLong() actually allocates about 1KB at a time, and will return NULL if malloc() can't find that much. OTOH, 1KB isn't big, and PyInt_FromLong(1) specifically should be returning a shared reference to a pre-existing PyIntObject (a number of small integer objects are constructed at Python initialization time, and PyInt_FromLong() returns references to them instead of allocating new memory). So PyInt_FromLong(1) should have taken this path: v = small_ints[ival + NSMALLNEGINTS]; But if a wild store had stuffed NULL into 1's slot in the small_ints vector, the *next* line would have blown up with a NULL-pointer dereference before PyInt_FromLong() could have returned: Py_INCREF(v); return (PyObject *) v; So, in all, it appears impossible for PyInt_FromLong(1) to return NULL. If it's reproducible, run it under a debugger and step into the errant PyInt_FromLong(1) to see what's happening? Could be a compiler optimization bug (while rare, they have happened in Python). From tim.peters at gmail.com Tue Dec 7 19:02:51 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 7 19:03:52 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <41B5E8AB.8060405@zope.com> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> Message-ID: <1f7befae04120710026748686c@mail.gmail.com> [Jim Fulton] > Ugh. Part of the problem is that all of those calls are unchecked, > Dang us. If they were checked, then, who knows, we might have > gotten informative exceptions. They certainly "should" be checked, but as a pragmatic matter PyInt_FromLong(1) can't fail -- Python allocates an int object for 1 (and for about 100 other popular little integers) when it starts up, and PyInt_FromLong() just returns a new reference to these pre-existing objects whenever possible. So, wrt: > I'd say the first step should be to add checks that's probably not going to help. I'd make it the fourth thing <wink>. From jim at zope.com Tue Dec 7 19:06:54 2004 From: jim at zope.com (Jim Fulton) Date: Tue Dec 7 19:07:00 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <1f7befae04120710026748686c@mail.gmail.com> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> Message-ID: <41B5F13E.5080901@zope.com> Tim Peters wrote: > [Jim Fulton] > >>Ugh. Part of the problem is that all of those calls are unchecked, >>Dang us. If they were checked, then, who knows, we might have >>gotten informative exceptions. > > > They certainly "should" be checked, but as a pragmatic matter > PyInt_FromLong(1) can't fail -- Python allocates an int object for 1 > (and for about 100 other popular little integers) when it starts up, > and PyInt_FromLong() just returns a new reference to these > pre-existing objects whenever possible. I know. I'm sure that's why we don't bother. But, obviously, it can fail. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From tim.peters at gmail.com Tue Dec 7 19:17:19 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 7 19:18:30 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <41B5F13E.5080901@zope.com> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> <41B5F13E.5080901@zope.com> Message-ID: <1f7befae04120710175ae5882@mail.gmail.com> [Tim] >> ... but as a pragmatic matter PyInt_FromLong(1) can't fail -- [Jim] > I know. I'm sure that's why we don't bother. But, obviously, > it can fail. I disagree -- it's not obvious at all. Looking at the code, it's far more likely that Andreas misunderstood the cause of the failure than that PyInt_FromLong(1) actually returned NULL. If it did return NULL, then it's got to be something as rare as bad code generation (for reasons I explained earlier), or a non-standard compilation that fiddled the NSMALLPOSINTS and/or NSMALLNEGINTS #defines to insane values. This is the entire expected path in PyInt_FromLong(1): PyObject * PyInt_FromLong(long ival) { register PyIntObject *v; #if NSMALLNEGINTS + NSMALLPOSINTS > 0 if (-NSMALLNEGINTS <= ival && ival < NSMALLPOSINTS) { v = small_ints[ival + NSMALLNEGINTS]; Py_INCREF(v); #ifdef COUNT_ALLOCS if (ival >= 0) quick_int_allocs++; else quick_neg_int_allocs++; #endif return (PyObject *) v; } #endif It's not possible for that to return NULL -- even if small_ints[] got corrupted, so that v == NULL, the Py_INCREF(v) would have blown up before the function could have returned. From amk at amk.ca Tue Dec 7 19:30:20 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Dec 7 19:30:41 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <1f7befae04120710026748686c@mail.gmail.com> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> Message-ID: <20041207183020.GA22351@rogue.amk.ca> On Tue, Dec 07, 2004 at 01:02:51PM -0500, Tim Peters wrote: > pre-existing objects whenever possible. So, wrt: > > I'd say the first step should be to add checks > that's probably not going to help. I'd make it the fourth thing <wink>. Is it possible that some other Python API call is raising an exception, but a NULL return isn't being checked for, and PyInt_FromLong() happens to be the first bit of code that does 'if (PyErr_Occurred())'? Though from a quick glance at PyInt_FromLong() and the macros it uses, I don't see any references to PyErr_Occurred()... --amk From lists at andreas-jung.com Tue Dec 7 19:34:32 2004 From: lists at andreas-jung.com (Andreas Jung) Date: Tue Dec 7 19:34:37 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <1f7befae04120710175ae5882@mail.gmail.com> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> <41B5F13E.5080901@zope.com> <1f7befae04120710175ae5882@mail.gmail.com> Message-ID: <47517AA25D790FF4C7D2A450@[192.168.0.102]> Sorry, false alarm :-( There assignment of the NULL occurs in the if-clause of the corresponding code (I have overseen the ASSIGN call): if (! PyInt_Check(p)) { if (PyDict_Check(p)) { if (PyString_Check(name) || PyUnicode_Check(name)) { ASSIGN(p, PyObject_GetItem(p, name)); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ if (p == NULL) { puts("PyObject returned NULL"); PyErr_Clear(); } } else p = PyInt_FromLong((long)1); ...doing some further investigations on that. -aj From tim.peters at gmail.com Tue Dec 7 19:40:01 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 7 19:40:16 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <20041207183020.GA22351@rogue.amk.ca> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> <20041207183020.GA22351@rogue.amk.ca> Message-ID: <1f7befae0412071040346940c8@mail.gmail.com> [A.M. Kuchling] > Is it possible that some other Python API call is raising an > exception, but a NULL return isn't being checked for, and > PyInt_FromLong() happens to be the first bit of code that does 'if > (PyErr_Occurred())'? Though from a quick glance at > PyInt_FromLong() and the macros it uses, I don't see any > references to PyErr_Occurred()... A long stare wouldn't uncover any either <wink>. This isn't it. That 1 is passed as an argument is important too, which cuts out all but the simplest path thru this code. The (current Zope 2.7 branch) source for cAccessControl.c contains six calls to PyInt_FromLong(). Five of them pass the literal 1. The other passes a variable with value -1, 0, or 1. So regardless of which of these Andreas is talking about, it's going thru PyInt_FromLong's fast path. From tim.peters at gmail.com Tue Dec 7 20:10:20 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 7 20:10:54 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <47517AA25D790FF4C7D2A450@192.168.0.102> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> <41B5F13E.5080901@zope.com> <1f7befae04120710175ae5882@mail.gmail.com> <47517AA25D790FF4C7D2A450@192.168.0.102> Message-ID: <1f7befae0412071110707ff1db@mail.gmail.com> [Andreas Jung] > Sorry, false alarm :-( There assignment of the NULL occurs in the > if-clause of the corresponding code (I have overseen the ASSIGN > call): Thanks for the followup! > if (! PyInt_Check(p)) > { > if (PyDict_Check(p)) > { > if (PyString_Check(name) || > PyUnicode_Check(name)) > { > ASSIGN(p, PyObject_GetItem(p, name)); > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > if (p == NULL) { > puts("PyObject returned NULL"); > PyErr_Clear(); > } > } > else > p = PyInt_FromLong((long)1); > > ...doing some further investigations on that. I note that all of this is nested inside another "if (p) {...}" block. That implies the "p = PyInt_FromLong((long)1);" line is at least a memory leak: it overwrites p without decref'ing p first. From jim at zope.com Tue Dec 7 20:19:43 2004 From: jim at zope.com (Jim Fulton) Date: Tue Dec 7 20:19:46 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <1f7befae0412071110707ff1db@mail.gmail.com> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> <41B5F13E.5080901@zope.com> <1f7befae04120710175ae5882@mail.gmail.com> <47517AA25D790FF4C7D2A450@192.168.0.102> <1f7befae0412071110707ff1db@mail.gmail.com> Message-ID: <41B6024F.4070201@zope.com> Tim Peters wrote: > [Andreas Jung] > >>Sorry, false alarm :-( There assignment of the NULL occurs in the >>if-clause of the corresponding code (I have overseen the ASSIGN >>call): > > > Thanks for the followup! > > >> if (! PyInt_Check(p)) >> { >> if (PyDict_Check(p)) >> { >> if (PyString_Check(name) || >>PyUnicode_Check(name)) >> { >> ASSIGN(p, PyObject_GetItem(p, name)); >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> if (p == NULL) { >> puts("PyObject returned NULL"); >> PyErr_Clear(); >> } >> } >> else >> p = PyInt_FromLong((long)1); >> >>...doing some further investigations on that. > > > I note that all of this is nested inside another "if (p) {...}" block. > That implies the "p = PyInt_FromLong((long)1);" line is at least a > memory leak: it overwrites p without decref'ing p first. The ASSIGN macro DECREFs it's first argument if it is non-NULL. It loosly models a Python assignment, assuming that it owns the reference to the second argument. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From tim.peters at gmail.com Tue Dec 7 21:25:06 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 7 21:25:19 2004 Subject: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL In-Reply-To: <41B6024F.4070201@zope.com> References: <9DFB75B2DC037BC6AAD05F9E@192.168.0.102> <41B5E8AB.8060405@zope.com> <1f7befae04120710026748686c@mail.gmail.com> <41B5F13E.5080901@zope.com> <1f7befae04120710175ae5882@mail.gmail.com> <47517AA25D790FF4C7D2A450@192.168.0.102> <1f7befae0412071110707ff1db@mail.gmail.com> <41B6024F.4070201@zope.com> Message-ID: <1f7befae04120712255df4861e@mail.gmail.com> >>> if (! PyInt_Check(p)) >>> { >>> if (PyDict_Check(p)) >>> { >>> if (PyString_Check(name) || PyUnicode_Check(name)) >>> { >>> ASSIGN(p, PyObject_GetItem(p, name)); >>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> >>> if (p == NULL) { >>> puts("PyObject returned NULL"); >>> PyErr_Clear(); >>> } >>> } >>> else >>> p = PyInt_FromLong((long)1); >>> [Tim] >> I note that all of this is nested inside another "if (p) {...}" block. >> That implies the "p = PyInt_FromLong((long)1);" line is at least a >> memory leak: it overwrites p without decref'ing p first. [Jim] > The ASSIGN macro DECREFs it's first argument if it is non-NULL. > > It loosly models a Python assignment, assuming that it owns the > reference to the second argument. ASSIGN isn't executed on the path in question. I really can't follow nesting with that indentation style. Reformatting in Python style makes it obvious to me: if (p) { if (! PyInt_Check(p)) { if (PyDict_Check(p)) { if (PyString_Check(name) || PyUnicode_Check(name)) { ASSIGN(p, PyObject_GetItem(p, name)); if (p == NULL) PyErr_Clear(); } else p = PyInt_FromLong(1); } else { ASSIGN(p, callfunction2(p, name, value)); if (p == NULL) goto err; } } } "p = PyInt_FromLong(1)" is in an ``else`` block. The only ASSIGN before it is in that ``else`` block's disjoint ``if`` block. From jlg at dds.nl Tue Dec 7 23:52:45 2004 From: jlg at dds.nl (Johannes Gijsbers) Date: Tue Dec 7 23:49:55 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_shutil.py, 1.10, 1.11 In-Reply-To: <2m4qiyqhtx.fsf@starship.python.net> References: <E1CbPoH-0005LW-Qo@sc8-pr-cvs1.sourceforge.net> <2m4qiyqhtx.fsf@starship.python.net> Message-ID: <20041207225245.GB9734@authsmtp.dds.nl> On Tue, Dec 07, 2004 at 02:54:50PM +0000, Michael Hudson wrote: > Is that really the best way to check for root? It may be, I guess, > but I'd have expected os.geteuid() to be more reliable... Ah yes, I suppose it is. I'll change it tomorrow night if no Unix-guru steps up with an alternative solution. > Or is it os.getuid()? I always get confused by these functions... Based on my reading of the difference between geteuid and getuid, I'd go for geteuid. Cheers, Johannes From tjreedy at udel.edu Wed Dec 8 02:41:45 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Dec 8 02:41:41 2004 Subject: [Python-Dev] Re: Rewriting PEP4 References: <002401c4dbf5$b847cc00$f301a044@oemcomputer> <41B556FE.20504@v.loewis.de> Message-ID: <cp5m4c$fkv$1@sea.gmane.org> ""Martin v. Löwis"" <martin@v.loewis.de> wrote in message news:41B556FE.20504@v.loewis.de... As a (currently) casual user of Python, this is my view of the standard library dilemma and its solution: 1. It is too small: I may someday want to use a module not yet added. 2. It is too big: I cannot keep everything in mind at once and cannot remember, without referring back to the lib manual or pydev, which modules are currently or slated to become deprecated. 3. I do not wish to have other people's code broken,or books made obsolete, without sufficient reason (ie, the code is actively dangerous versus merely broken or superceded). My preferred solution is to apply the 'out of sight, out of mind' principle. 1. Move obsolete modules to a separate directory (lib_old sounds fine) and put that directory in pythonpath. When I ran Python (1.3) from a 20 meg disk, I would have preferred complete removal, but with 60+ gigs, the small extra space is no longer an issue. If someone needs the space or wants to guarantee non-use of old modules, deleting lib_old is easy enough. 2. Remove the docs for obsolete modules from the main part of the current lib reference. Sub-choices for legacy chapters are either complete removal or segregation into a separate document or apppendix to current one. Terry J. Reedy From gvanrossum at gmail.com Wed Dec 8 23:18:48 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed Dec 8 23:18:50 2004 Subject: [Python-Dev] 2.4 news reaches interesting places Message-ID: <ca471dc204120814186f5dc423@mail.gmail.com> I was pleasantly surprised to find a pointer to this article in a news digest that the ACM emails me regularly (ACM TechNews). http://gcn.com/vol1_no1/daily-updates/28026-1.html One thing that bugs me: the article says 3 or 4 times that Python is slow, each time with a refutation ("but it's so flexible", "but it's fast enough") but still, they sure seem to harp on the point. This is a PR issue that Python needs to fight -- any ideas? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Dec 8 23:39:31 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Dec 8 23:39:34 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> At 02:18 PM 12/8/04 -0800, Guido van Rossum wrote: >I was pleasantly surprised to find a pointer to this article in a news >digest that the ACM emails me regularly (ACM TechNews). > >http://gcn.com/vol1_no1/daily-updates/28026-1.html > >One thing that bugs me: the article says 3 or 4 times that Python is >slow, each time with a refutation ("but it's so flexible", "but it's >fast enough") but still, they sure seem to harp on the point. This is >a PR issue that Python needs to fight -- any ideas? The only thing that will fix the PR issue is to have a Python compiler distributed as part of the language. It doesn't matter if it doesn't support the full generality of Python, or even if it doesn't speed many operations up much. The only real requirements are that it can be used to produce "native" executables, and that it be an official part of the language, not a separately-distributed tool like Psyco or Pyrex. Then, it will perhaps be a sufficient "security blanket" to stop people FUDding about it. I imagine you could speed up the Python interpreter until it's faster than half the Java JIT's out there, and people will still ask, "But can I compile to an .exe?" On the other hand, if you add a compiler, we'll see articles like the above talking about how Python can now be compiled and so therefore it's suitable for all kinds of things it wasn't before. :) Of course, it would be *really* useful if the compiler were coupled with optional type declarations for Python-the-language, because then we could ultimately dispense with the current syntax warts of Pyrex (and the runtime/distribution warts of ctypes) for interfacing with C. But I don't think that having the compiler actually be useful is a prerequisite for solving the PR issue. :) From pythondev at bitfurnace.com Wed Dec 8 23:59:05 2004 From: pythondev at bitfurnace.com (damien morton) Date: Wed Dec 8 23:59:11 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <41B78739.1080200@bitfurnace.com> Guido van Rossum wrote: > I was pleasantly surprised to find a pointer to this article in a news > digest that the ACM emails me regularly (ACM TechNews). > > http://gcn.com/vol1_no1/daily-updates/28026-1.html > > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? In public documents - documentation, release notes, etc etc.: Remove all references to the performance of Python. From now on, Python has always been extremely fast, has no need to get any faster, and any speed increases are of so little importance that they barely rate a mention (no percentage increases mentioned in release notes). If you must mention performance, do it in terms of the use of C modules such as numarray, and for benchmarks that other languages cant compete in. Write a benchmark which uses common C extensions, that Java cant compete with. Replace all references to a 'Python interpreter' with 'Python Virtual Machine' Ensure that the terms 'Python compiler' and 'compiled python' are liberally sprinkled around. Start emphasising the compilation step from python source to python bytecodes instead of making it transparent. From sxanth at ceid.upatras.gr Thu Dec 9 00:25:20 2004 From: sxanth at ceid.upatras.gr (Stelios Xanthakis) Date: Thu Dec 9 00:25:24 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> Message-ID: <Pine.GSO.4.61.0412090113520.15784@zenon.ceid.upatras.gr> > The only thing that will fix the PR issue is to have a Python compiler > distributed as part of the language. It doesn't matter if it doesn't > support the full generality of Python, or even if it doesn't speed many > operations up much. The only real requirements are that it can be used > to produce "native" executables I don't hink it's a matter of native executables. Hopefully all the real algorithms are already in native executables as builtins (dictionary lookup, etc). And generally, one can wrap all the tough code in C and take advantage of the flexibility of python at a higher level. pygame is the best example of this! Basic stuff from complexity theory: the real complex parts are few and isolated in C. For the other we can take advantage of a higher level dynamic language. Stelios From nas at arctrix.com Thu Dec 9 00:29:09 2004 From: nas at arctrix.com (Neil Schemenauer) Date: Thu Dec 9 00:29:13 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <20041208232909.GA6354@mems-exchange.org> On Wed, Dec 08, 2004 at 02:18:48PM -0800, Guido van Rossum wrote: > This is a PR issue that Python needs to fight -- any ideas? I'm not good at PR so I will continue to try to make it faster. In my copious free time I plan to: * finish the AST compiler (no performance benefit but makes other things easier) * optimize module namespaces as per the PEPs (whose numbers I cannot recall) * optimize function/method invocation (we must be able to do something better here, they are currently very expensive). Fredrik has mentioned "call-site caching". * write a new, register based VM From hpk at trillke.net Thu Dec 9 01:39:38 2004 From: hpk at trillke.net (holger krekel) Date: Thu Dec 9 01:39:40 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <20041209003938.GE4708@solar.trillke.net> [Guido van Rossum Wed, Dec 08, 2004 at 02:18:48PM -0800] > I was pleasantly surprised to find a pointer to this article in a news > digest that the ACM emails me regularly (ACM TechNews). > > http://gcn.com/vol1_no1/daily-updates/28026-1.html > > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? What about doing a survey on c.l.py asking questions like: what do you find runs slowly in Python that should run faster? Could you help with some concrete - preferably real life examples with speed problems?". If python-dev takes the time to seriously debate (and debunk :-) upcoming code and suggestions then such a process could be very useful for all sides and also for PR purposes. IMO the biggest PR problem is that python programmers themselves [*] tend to say that Python is slow and it's interesting to find out why they think so, document and discuss the "answers" if any. I am not saying that such questioning/discussion doesn't already happen sometimes. But doing a survey in a more systematic way might let us find out how pervasive the perception of "Python is too slow" really is. Maybe it turns out that not many people have concrete problems to offer? Anyway, this would probably also be interesting for the various alternative Python implementations currently in the works. just an idea, holger [*] For example, something i stumbled upon today: http://unununium.org/articles/languages where it states (without providing any details!): But what about that fast system? Python isn't a slow language; it just has a slow implementation. There are many projects underway to correct this situation: Psyco, PyPy, Starkiller, IronPython, and Parrotcode are among them. It's likely these projects will be nearing completion when the time comes for Unununium to look at optimizations. From pje at telecommunity.com Thu Dec 9 02:30:02 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Dec 9 02:30:11 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <Pine.GSO.4.61.0412090113520.15784@zenon.ceid.upatras.gr> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> At 01:25 AM 12/9/04 +0200, Stelios Xanthakis wrote: >>The only thing that will fix the PR issue is to have a Python compiler >>distributed as part of the language. It doesn't matter if it doesn't >>support the full generality of Python, or even if it doesn't speed many >>operations up much. The only real requirements are that it can be used >>to produce "native" executables > >I don't hink it's a matter of native executables. As I explained later in that message, "native" simply means, "has an .exe extension on Windows". For PR purposes, it would suffice to bundle py2exe with Python 2.5 and say that Python "now includes a compiler to produce executable files". This will then be picked up and misinterpreted by the trade press in exactly the same way that the article Guido cited picked up and misinterpreted what was said about 2.4. If you read the article carefully, you will notice that the author translated "we rewrote a few modules in C" into "we made Python faster by switching to C". If you ask anybody what language is faster, language X or C, most everybody will answer "C", regardless of what X is (unless it's assembly, of course). All of the discussion about *actually* improving Python's performance is moot for PR purposes. Public perception is not swayed by mere facts (as one might cynically observe of the U.S. political system). If the goal is to achieve a PR win, the important thing is to pick a meme that's capable of succeeding, and stay "on message" with it. The *only* meme that's organically capable of trumping "Python is slow because it's interpreted" is "Python is compiled now". Me, I don't really care one way or the other. I used to sell software I wrote in TRS-80 Basic, so Python's performance is fine for me, and I'm certainly not a compiler bigot. I'm just responding to Guido's inquiry about what might work to increase Python's *perceived* speed in popular consciousness, not its actual speed. From andrew at indranet.co.nz Thu Dec 9 03:32:10 2004 From: andrew at indranet.co.nz (Andrew McGregor) Date: Thu Dec 9 03:32:28 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz> Well, for a lot of applications for Python, the performance that really counts is time from no code but a pile of data to code and processed data. Python shines at that because nearly always the time to write the code dominates, so it doesn't matter what the run time is. I wrote a little tool to translate a bunch of free data into scenery for the X-Plane flight simulator. If I'd tried to do it in C, I'd still be debugging it, whereas I released it and the scenery I was building six months ago. The run time of the C version would be, I speculate, about 5 times faster than the Python (given that psyco speeds it up about 8 times, and I would not have been so fancy with the algorithms in C). But a 5x improvement on a 24 hour runtime is not 6 months of improvement. The other thing we can do is finish the portable backend for psyco and make it a standard module. Then Python won't be slow, it will be compiled, and py2exe will be able to make a single-file executable. Andrew On 9/12/2004, at 11:18 AM, Guido van Rossum wrote: > I was pleasantly surprised to find a pointer to this article in a news > digest that the ACM emails me regularly (ACM TechNews). > > http://gcn.com/vol1_no1/daily-updates/28026-1.html > > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ > andrew%40indranet.co.nz > > From ilya at bluefir.net Thu Dec 9 04:57:25 2004 From: ilya at bluefir.net (Ilya Sandler) Date: Thu Dec 9 04:55:08 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <Pine.LNX.4.58.0412081904450.1012@bagira> On Wed, 8 Dec 2004, Guido van Rossum wrote: > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? > On PR side of the issue one idea would be to avoid using the words interpreted/intepreter less and use the words compiler/byte-compilation/virtual machine instead... On non-PR side of the issue (I do think that python slowness is a real problem for at least some people/domains): would bundling of Psyco with Python be a good idea? Ilya From python at rcn.com Thu Dec 9 04:55:59 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Dec 9 05:00:16 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? * Ask a prominent PSF member, ESR, to stop bagging on that point in his book (python is much faster since he wrote it and Moore's law has made it less of an issue over time). Py1.5.2 running on a 133Mhz 386 is not the same beast as well designed Py2.4 code running on a 3Ghz Pentium IV (also I would argue that many apps are computationally bound anyway). * Have python.org prominently feature an article of Python's use in high-performance environments. IIRC, somebody wrote a realtime voice over internet system and found that with good design, there was no speed issue. Also, the cellphone apps may make a good example. * Another prominent developer and PSF member wrote a much publicized, unfavorable log entry basically saying that he was unimpressed with Py2.4. While that log entry was subsequently revised, it did serve as anti-advocacy. Also, my request was denied for better presentation of performance related benchmarks in the widely read What's New document. Right now, it features somewhat unimpressive and misleading pystone results. Were we to show comparative pybench results 2.0 to 2.1 to 2.2 to 2.3 to 2.4, it would become clear that some of the performance concerns are out of date. Parrotbench and test_decimal also reveal some modest gains over 2.3. * A ZDNet reporter had setup a phone interview with me but did not follow through. I had planned to nip the issue in the bud by focusing on the Py2.4's vastly improved scalability: - The trend toward use of iterators, generators, generator expressions, and itertools scale-up well because of their superb memory performance and ability to save state. Apps using generators and genexps save all the time that was formerly lost to accessing instance variables. And apps keying off of itertools can sometimes completely avoid interpreter overhead. IOW, Py2.4 provides a strong toolset for writing clean, high-performance apps with a minimum of development time. - Behind the scenes, almost every important building block has either improved algorithms, memory optimizations, speed optimizations, and/or has been rewritten in C (while still preserving their pure Python equivalents). The bytecodes get optimized for speed and readability, and the eval-loop itself is tighter. Lists initialize, append, pop, and extend much faster and are more conservative with memory. List comps are up to twice as fast. Dictionary iteration is speedier. Sets, bisect, and heaps now have C implementations like other basic building blocks. All of the fifo queues in the library now use an O(1) algorithm instead of O(n) -- this improves scalability for everything from threading and queues to mutexs and asynchat. * Any PR effort should also emphasize that no usability trade-offs were made along the way. A number of features make Py2.4 easier to use than 1.5.6: list comps, genexps, generators, sets, nested scopes, int/long unification, decorators, etc. Some might argue that it takes a modicum effort to learn listcomps, generators, and decorators, but they cannot make any bones about the cleanliness, simplicity, beauty, and speed of the resulting code. Python will always lead in maintainability and development time. * Python's website has traditionally been self-centered, leaving others to have to make the case for their own products. Perhaps, it is time to change that. Those who really care about speed cannot make a balanced decision about Python without considering psyco, pyrex, numpy, and the like as part of the total development environment. * There is another angle that is difficult to present but a reality never-the-less. It is easy to write toy C / C++ code that is blazingly fast. However, when you add error checking, fault tolerance, interaction with real world, etc, the code gets slower. Also because that code is harder to write, it is somewhat common to use crummy algorithms (linear searches for example). I really liked Tim's comment that code using Python's dictionaries runs 10,000 times faster than anything written in any other language. That seems to get to the point across well enough. * A bit of good news is that the upcoming Python cookbook will make a strong case for Py2.4 being a tool of choice for those who care speed, space, maintainability, and development time. It is a nice, highly visible bit of advocacy. * One last thought. Articles will always pick on something. It's not really a terrible thing to have someone say the superb productivity gains come at the price of running slower than C. I would much rather hear that than have people bag on the docs or standard library or launch into a diatribe @decocrator destroying the beauty of the language. Raymond From mdehoon at ims.u-tokyo.ac.jp Thu Dec 9 05:47:01 2004 From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon) Date: Thu Dec 9 05:43:46 2004 Subject: [Python-Dev] PyOS_InputHook enhancement proposal Message-ID: <41B7D8C5.9000008@ims.u-tokyo.ac.jp> PyOS_InputHook is a pointer to a function that is called periodically (ten times per second) while Python is idle, for example, when waiting for a user command. Python C extension modules can set this pointer to a hook function defined in the extension module. For example, _tkinter.c makes use of PyOS_InputHook to get messages delivered to its widgets. A problem arises when two or more extension modules want to set PyOS_InputHook. For example, the scientific plotting package pygist needs PyOS_InputHook to get messages delivered to its graphics windows, and may therefore conflict with the Python GUI IDLE, which uses Tkinter. Chaining won't work, as it will fail when an extension module wants to remove its hook function. My suggestion is therefore to replace PyOS_InputHook by two functions PyOS_AddInputHook and PyOS_RemoveInputHook, and let Python keep track of which hooks are installed. This way, an extension module can add a hook function without having to worry about other extension modules trying to use the same hook. Any comments? Would I need to submit a PEP for this proposal? --Michiel, U Tokyo. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon From python at rcn.com Thu Dec 9 05:43:49 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Dec 9 05:48:03 2004 Subject: [Python-Dev] The other Py2.4 issue Message-ID: <001401c4dda9$ad605680$e841fea9@oemcomputer> Acceptance for Py2.4 partially hinges on how quickly third party apps have their binaries updated. I wonder if there is anything we can do to help. Raymond -----Original Message----- From: image-sig-bounces@python.org [mailto:image-sig-bounces@python.org] On Behalf Of Tuure Laurinolli Sent: Tuesday, December 07, 2004 2:51 AM To: image-sig@python.org Subject: [Image-SIG] Re: Python 2.4 Spencer Ogden wrote: > I was wondering if there was a time frame for a windows binary for Py > 2.4. Is it possible to compile the source against 2.4 myself? There didn't seem to be any problems compiling it against 2.4. I managed to get some version up and running, but problems with the required shared libraries(I seem to have many, incompatible versions of jpeg around...) killed my enthusiasm and I returned to 2.3. _______________________________________________ Image-SIG maillist - Image-SIG@python.org http://mail.python.org/mailman/listinfo/image-sig From phd at mail2.phd.pp.ru Thu Dec 9 11:17:02 2004 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu Dec 9 11:17:05 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> Message-ID: <20041209101702.GA12506@phd.pp.ru> On Wed, Dec 08, 2004 at 08:30:02PM -0500, Phillip J. Eby wrote: > As I explained later in that message, "native" simply means, "has an .exe > extension on Windows". And very soon that strategy will backfire - people will start PRing "but those .exe's are nothing more than a python interpreter in disguise" which in my opnion is even worse. > All of the discussion about *actually* improving Python's performance is > moot for PR purposes. Hence we must stop spending our very valuable time thinking about PR and return to actually improving Python (not only performance). > If the goal is > to achieve a PR win, the important thing is to pick a meme that's capable > of succeeding, and stay "on message" with it. Translating to a plain langauge: "PSF should spend money spreading around a counter-PR". I am afraid PSF doesn't have enough money, and even if it has - should we really run down that path? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at mail2.phd.pp.ru Thu Dec 9 11:22:50 2004 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu Dec 9 11:22:52 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> Message-ID: <20041209102250.GB12506@phd.pp.ru> On Wed, Dec 08, 2004 at 10:55:59PM -0500, Raymond Hettinger wrote: > * Have python.org prominently feature an article of Python's use in > high-performance environments. IIRC, somebody wrote a realtime voice > over internet system and found that with good design, there was no speed > issue. Also, the cellphone apps may make a good example. +Games. > * Python's website has traditionally been self-centered, leaving others > to have to make the case for their own products. Perhaps, it is time to > change that. Those who really care about speed cannot make a balanced > decision about Python without considering psyco, pyrex, numpy, and the > like as part of the total development environment. That's overreaction, I think. People always say this and that about python - python is slow, and python is not like java, and python does not have static type checks... And what? Should the webmasters adapt the site to every complain? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From aahz at pythoncraft.com Thu Dec 9 14:15:55 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Dec 9 14:15:58 2004 Subject: [Python-Dev] PyOS_InputHook enhancement proposal In-Reply-To: <41B7D8C5.9000008@ims.u-tokyo.ac.jp> References: <41B7D8C5.9000008@ims.u-tokyo.ac.jp> Message-ID: <20041209131555.GB10465@panix.com> On Thu, Dec 09, 2004, Michiel Jan Laurens de Hoon wrote: > > My suggestion is therefore to replace PyOS_InputHook by two functions > PyOS_AddInputHook and PyOS_RemoveInputHook, and let Python keep track of > which hooks are installed. This way, an extension module can add a hook > function without having to worry about other extension modules trying > to use the same hook. > > Any comments? Would I need to submit a PEP for this proposal? Because this is only for the C API, your best bet is to write a patch and submit it to SF. If people whine or it gets rejected, then write a PEP. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From stephen.kennedy at havok.com Thu Dec 9 14:20:38 2004 From: stephen.kennedy at havok.com (Stephen Kennedy) Date: Thu Dec 9 14:32:15 2004 Subject: [Python-Dev] How to install tile (or any other tcl module) Message-ID: <loom.20041209T141319-804@post.gmane.org> I've been trying to get Tile to work with python. It can make your tkinter apps look like http://tktable.sourceforge.net/tile/screenshots/demo-alt-unix.png See http://tktable.sourceforge.net/tile/ Under linux I built tile from source, installed and it just works. import Tkinter root = Tkinter.Tk() root.tk.call('package', 'require', 'tile') root.tk.call('namespace', 'import', '-force', 'ttk::*') root.tk.call('tile::setTheme', 'alt') ### Widgets are now pretty! Under win32, I installed the binary package into python/tcl (i.e. python/tcl/tile0.5) with all the other tcl packages, but tcl can't find it. Any ideas? Traceback (most recent call last): File "Script1.py", line 5, in ? root.tk.call('package', 'require', 'tile') _tkinter.TclError: can't find package tile Stephen. From anthony at interlink.com.au Thu Dec 9 14:36:16 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Dec 9 14:36:33 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> References: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> Message-ID: <200412100036.17253.anthony@interlink.com.au> On Thursday 09 December 2004 14:55, Raymond Hettinger wrote: > * Have python.org prominently feature an article of Python's use in > high-performance environments. IIRC, somebody wrote a realtime voice > over internet system and found that with good design, there was no speed > issue. Also, the cellphone apps may make a good example. That was me. I gave a keynote (45 minute) version of the talk last week at OSDC, and I believe it was videoed and will be available eventually. This is good propaganda. Anthony -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood. From Scott.Daniels at Acm.Org Thu Dec 9 15:07:41 2004 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Thu Dec 9 15:06:19 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <20041209102250.GB12506@phd.pp.ru> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> <20041209102250.GB12506@phd.pp.ru> Message-ID: <cp9m4m$isn$1@sea.gmane.org> Oleg Broytmann wrote: > Raymond Hettinger wrote: >>* Python's website has traditionally been self-centered, leaving others >>to have to make the case for their own products. Perhaps, it is time to >>change that. Those who really care about speed cannot make a balanced >>decision about Python without considering psyco, pyrex, numpy, and the >>like as part of the total development environment. > > That's overreaction, I think. People always say this and that about > python - python is slow, and python is not like java, and python does > not have static type checks... And what? Should the webmasters adapt the > site to every complain? > Perhaps a link on the main page to a "for even more speed" page where we can combine advice on how to improve application performance (measure, abstract, choose good data structures) with references to these other projects for particular applications. This is the place to say things like "operating on every character of a string is seldom efficient." -- -- Scott David Daniels Scott.Daniels@Acm.Org From phd at mail2.phd.pp.ru Thu Dec 9 15:11:04 2004 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu Dec 9 15:11:07 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <cp9m4m$isn$1@sea.gmane.org> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> <20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org> Message-ID: <20041209141104.GA18348@phd.pp.ru> On Thu, Dec 09, 2004 at 06:07:41AM -0800, Scott David Daniels wrote: > Oleg Broytmann wrote: > > That's overreaction, I think. > > Perhaps a link on the main page Why on the main page? There are Topics Guide at http://python.org/topics/ that describes the ways Python can be used in some popular areas. Let's add another topic, "Making things fast". Let's even make it the first topic, though I personnaly dont see a need for this. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From anthony at interlink.com.au Thu Dec 9 15:17:50 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Dec 9 15:18:49 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <20041209141104.GA18348@phd.pp.ru> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cp9m4m$isn$1@sea.gmane.org> <20041209141104.GA18348@phd.pp.ru> Message-ID: <200412100117.50910.anthony@interlink.com.au> FWIW, I was planning on writing a tutorial (working title: "Making Python Code Not Suck") for some conference or another... talked to a bunch of people last week at OSDC, and it seems like something people are interested in. Got a bunch of stuff already down from various notes I've written in the past when helping coworkers and the like. Anthony -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood. From aahz at pythoncraft.com Thu Dec 9 15:55:40 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Dec 9 15:55:54 2004 Subject: [Python-Dev] How to install tile (or any other tcl module) In-Reply-To: <loom.20041209T141319-804@post.gmane.org> References: <loom.20041209T141319-804@post.gmane.org> Message-ID: <20041209145540.GB13555@panix.com> On Thu, Dec 09, 2004, Stephen Kennedy wrote: > > I've been trying to get Tile to work with python. > It can make your tkinter apps look like > http://tktable.sourceforge.net/tile/screenshots/demo-alt-unix.png > See http://tktable.sourceforge.net/tile/ Sorry, this is not a good place to get Python support; python-dev is for people actively working on Python language development. Please use comp.lang.python or one of the other resources. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From pje at telecommunity.com Thu Dec 9 16:30:10 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Dec 9 16:30:50 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <20041209101702.GA12506@phd.pp.ru> References: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com> At 01:17 PM 12/9/04 +0300, Oleg Broytmann wrote: >On Wed, Dec 08, 2004 at 08:30:02PM -0500, Phillip J. Eby wrote: > > As I explained later in that message, "native" simply means, "has an .exe > > extension on Windows". > > And very soon that strategy will backfire - people will start PRing >"but those .exe's are nothing more than a python interpreter in >disguise" which in my opnion is even worse. ISTR that for a long time, Visual Basic actually did the same thing. A few magazines mentioned the fact, but nobody really cared. However, if this is really a concern, bundle Pyrex as well. Both Pyrex and py2exe are distutils-based, so at that point you have a complete solution, including the "C" meme as well as the ".exe" meme. > > If the goal is > > to achieve a PR win, the important thing is to pick a meme that's capable > > of succeeding, and stay "on message" with it. > > Translating to a plain langauge: "PSF should spend money spreading >around a counter-PR". I am afraid PSF doesn't have enough money, and >even if it has - should we really run down that path? I'm not suggesting spending any money. Heck, I'm not suggesting *doing* anything. I just answered Guido's question about PR. Whether anybody wants to *do* anything about it is an entirely separate issue from discussing *what* would need to be done, if something was going to be done. From gmccaughan at synaptics-uk.com Thu Dec 9 17:39:21 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu Dec 9 17:39:53 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> Message-ID: <200412091639.21596.gmccaughan@synaptics-uk.com> On Wednesday 2004-12-08 22:39, Phillip J. Eby wrote: [Guido:] >> One thing that bugs me: the article says 3 or 4 times that Python is >> slow, each time with a refutation ("but it's so flexible", "but it's >> fast enough") but still, they sure seem to harp on the point. This is >> a PR issue that Python needs to fight -- any ideas? [Philip:] > The only thing that will fix the PR issue is to have a Python compiler > distributed as part of the language. It doesn't matter if it doesn't > support the full generality of Python, or even if it doesn't speed many > operations up much. The only real requirements are that it can be used to > produce "native" executables, and that it be an official part of the > language, not a separately-distributed tool like Psyco or Pyrex. Then, it > will perhaps be a sufficient "security blanket" to stop people FUDding > about it. Unfortunately, this may not be enough. Consider, by way of counterexample, Common Lisp, which - is compiled to native code - has optional type declarations - actually *does* run fast when compiled - has had all these properties for years and years but is still almost universally decried as "slow" by people who have never actually used it. It's true that it doesn't (as part of the standard, and in the free implementations I know of) have the ability to generate standalone executables with filenames ending in ".exe"; perhaps that's the key thing. The other thing that might work is to change the name of the language to "C" plus optional punctuation. -- g From janssen at parc.com Thu Dec 9 17:56:09 2004 From: janssen at parc.com (Bill Janssen) Date: Thu Dec 9 17:56:57 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: Your message of "Thu, 09 Dec 2004 08:39:21 PST." <200412091639.21596.gmccaughan@synaptics-uk.com> Message-ID: <04Dec9.085613pst."58617"@synergy1.parc.xerox.com> > The other thing that might work is to change the name of the language > to "C" plus optional punctuation. You mean "C@@" (pronounced C-pie-pie)? Bill From fredrik at pythonware.com Thu Dec 9 18:57:10 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu Dec 9 18:54:56 2004 Subject: [Python-Dev] Re: How to install tile (or any other tcl module) References: <loom.20041209T141319-804@post.gmane.org> <20041209145540.GB13555@panix.com> Message-ID: <cpa3hb$29m$1@sea.gmane.org> >> I've been trying to get Tile to work with python. >> It can make your tkinter apps look like >> http://tktable.sourceforge.net/tile/screenshots/demo-alt-unix.png >> See http://tktable.sourceforge.net/tile/ > > Sorry, this is not a good place to get Python support; python-dev is for > people actively working on Python language development. Please use > comp.lang.python or one of the other resources. in this case, http://mail.python.org/mailman/listinfo/tkinter-discuss is the right forum (also see: http://tkinter.unpythonic.net/wiki/TkinterDiscuss ) </F> From fumanchu at amor.org Thu Dec 9 19:21:00 2004 From: fumanchu at amor.org (Robert Brewer) Date: Thu Dec 9 19:23:07 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places Message-ID: <3A81C87DC164034AA4E2DDFE11D258E324538E@exchange.hqamor.amorhq.net> Anthony Baxter wrote: > FWIW, I was planning on writing a tutorial (working title: > "Making Python Code Not Suck") for some conference > or another... Perhaps, given your high profile in the Python developer community, you might reconsider the title? Little details like that are what PR is made of. Imagine Bush's next Executive Order being titled "Making American [Business|Military|People] Not Suck"... ;) Robert Brewer MIS Amor Ministries fumanchu@amor.org From mwh at python.net Thu Dec 9 19:28:01 2004 From: mwh at python.net (Michael Hudson) Date: Thu Dec 9 19:28:03 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E324538E@exchange.hqamor.amorhq.net> (Robert Brewer's message of "Thu, 9 Dec 2004 10:21:00 -0800") References: <3A81C87DC164034AA4E2DDFE11D258E324538E@exchange.hqamor.amorhq.net> Message-ID: <2mk6rrpbri.fsf@starship.python.net> "Robert Brewer" <fumanchu@amor.org> writes: > Anthony Baxter wrote: >> FWIW, I was planning on writing a tutorial (working title: >> "Making Python Code Not Suck") for some conference >> or another... > > Perhaps, given your high profile in the Python developer community, you > might reconsider the title? Little details like that are what PR is made > of. Imagine Bush's next Executive Order being titled "Making American > [Business|Military|People] Not Suck"... ;) Anthony's Australian, people expect this sort of thing from him :) Cheers, mwh -- Also, remember to put the galaxy back when you've finished, or an angry mob of astronomers will come round and kneecap you with a small telescope for littering. -- Simon Tatham, ucam.chat, from Owen Dunn's review of the year From pje at telecommunity.com Thu Dec 9 19:31:15 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Dec 9 19:32:05 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <200412091639.21596.gmccaughan@synaptics-uk.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com> At 04:39 PM 12/9/04 +0000, Gareth McCaughan wrote: >On Wednesday 2004-12-08 22:39, Phillip J. Eby wrote: > > The only thing that will fix the PR issue is to have a Python compiler > > distributed as part of the language. It doesn't matter if it doesn't > > support the full generality of Python, or even if it doesn't speed many > > operations up much. The only real requirements are that it can be used to > > produce "native" executables, and that it be an official part of the > > language, not a separately-distributed tool like Psyco or Pyrex. Then, it > > will perhaps be a sufficient "security blanket" to stop people FUDding > > about it. > >Unfortunately, this may not be enough. Consider, by way of counterexample, >Common Lisp, which > - is compiled to native code > - has optional type declarations > - actually *does* run fast when compiled > - has had all these properties for years and years >but is still almost universally decried as "slow" by people who have >never actually used it. It's true that it doesn't (as part of the >standard, and in the free implementations I know of) have the ability >to generate standalone executables with filenames ending in ".exe"; >perhaps that's the key thing. Among the great unwashed masses of Windows programmers, it probably is. Look at the marketing for virtually any off-beat commercial language for Windows (that's still around to look at), and you will usually find a prominent bullet point about how it makes .exe's. This is an incredibly powerful meme, though I don't entirely understand why. I do agree that it still may not be enough; I'm just saying that it's not a coincidence that so many development tools for non-C languages on Windows all ended up deciding to feature that bullet point. My theory is that it's because it's a FAQ, and it's often found on the official FAQ's of open source languages as well. Granted, the scope of the question is much broader than just speed, since it also encompasses application distribution issues, and the social status of the developer. One of the subtexts of the question is, "Can I make a *real* program with this thing?", or put more honestly, "Will other programmers laugh at me when they see my program isn't an .exe?" >The other thing that might work is to change the name of the language >to "C" plus optional punctuation. Well, we could always just call it CPython, and bill it as a faster C implementation of that popular dynamic language for Java, Jython. That way, we can also leverage the meme that C is faster than Java. :) Isn't it funny, by the way, that people don't go around talking about how slow Jython is? At least, I haven't seen it to the extent that I've seen with CPython. From foom at fuhm.net Thu Dec 9 20:16:18 2004 From: foom at fuhm.net (James Y Knight) Date: Thu Dec 9 20:16:24 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com> Message-ID: <CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net> On Dec 9, 2004, at 1:31 PM, Phillip J. Eby wrote: > Isn't it funny, by the way, that people don't go around talking about > how slow Jython is? At least, I haven't seen it to the extent that > I've seen with CPython. People talk about how slow CPython is, is because they are writing code targeted for it that traditionally would have been written in C. Python *is* slower than C. You can try to deny it, but it is simple fact. However, python is a really nice language to program in. So people use it anyways. It is "fast enough" for many things. Nobody talks about how slow Jython is, because nobody(1) is writing entire programs in it. Jython is used as a way to script Java programs, and as such, it is pretty much completely unimportant how fast it runs. Java has also traditionally got a lot of bad press for its lack of speed, and it was usually of the "not fast enough" variety, which is much worse than what's said of Python. However, a huge amount of progress has been made and Java really is quite speedy these days. It was not PR that made it faster, but hard work on the part of Sun. There is currently a lot of work going on in the python community to make Python faster, which indicates both that Python is too slow for some people, and that it will be faster in the future. This is a good thing, and not something that should be shoved under the rug to try to pretend python is ultra mega super fast already. James 1) Of course someone probably is, but you get the idea. From tdelaney at avaya.com Thu Dec 9 20:25:54 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Dec 9 20:27:14 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com> Michael Hudson wrote: > Anthony's Australian, people expect this sort of thing from him :) As another Australian, I think that "Making Python Not Suck" implies that if you don't do extra things, Python Sucks. This is not a good thing IMO. "Making Python Suck Less" would be even worse. How about "Python - You Can Have Your Cake And Eat It Too". Tim Delaney From jhylton at gmail.com Thu Dec 9 20:56:41 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Thu Dec 9 20:56:45 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com> <CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net> Message-ID: <e8bf7a5304120911564cef0c53@mail.gmail.com> On Thu, 9 Dec 2004 14:16:18 -0500, James Y Knight <foom@fuhm.net> wrote: > On Dec 9, 2004, at 1:31 PM, Phillip J. Eby wrote: > > Isn't it funny, by the way, that people don't go around talking about > > how slow Jython is? At least, I haven't seen it to the extent that > > I've seen with CPython. > > People talk about how slow CPython is, is because they are writing code > targeted for it that traditionally would have been written in C. Python > *is* slower than C. You can try to deny it, but it is simple fact. > However, python is a really nice language to program in. So people use > it anyways. It is "fast enough" for many things. I think you are exactly right. It's easy to get the impression that Python is slow when many programs written in Python *are* slowly than the same program written in C, C++, or Java. If you look are other "dynamic" languages, you'll find that IronPython is often faster than CPython and that Smalltalk and Common Lisp are often substantially faster than Python. Python isn't remarkably slow for a scripting language, but it is not fast. > There is currently a lot of work going on in the python community to > make Python faster, which indicates both that Python is too slow for > some people, and that it will be faster in the future. This is a good > thing, and not something that should be shoved under the rug to try to > pretend python is ultra mega super fast already. I agree, although it's not clear to me how much faster it will be in the future. Making a *fast* Python based on our own virtual execution environment (as opposed to piggybacking a JVM or CLR) is a big project. It's not clear than anyone has enough resources to make credible effort, so we're left to piecemeal improvements. The improvement I see from 2.2 to 2.4 on my box is about 55% (measured using pystone) over almost three years. I think the right thing to do with PR is frame the question differently. We need to say that people use Python because it lets them right clear, simple code or that it is easy to maintain or that dynamic languages are excellent for prototyping -- whatever the message is -- and simply avoid talking about speed. There are a lot of issues that affect the selection of a programming language, and speed is not always the most important one. It is easy for a journalist to write about and it is easy to report (possibly meaningless) measurements of speed. Jeremy From s.percivall at chello.se Thu Dec 9 21:10:02 2004 From: s.percivall at chello.se (Simon Percivall) Date: Thu Dec 9 21:10:07 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <cp9m4m$isn$1@sea.gmane.org> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> <20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org> Message-ID: <4FA61149-4A1E-11D9-9ADB-0003934AD54A@chello.se> On 2004-12-09, at 15.07, Scott David Daniels wrote: > Oleg Broytmann wrote: >> Raymond Hettinger wrote: >>> * Python's website has traditionally been self-centered, leaving >>> others >>> to have to make the case for their own products. Perhaps, it is >>> time to >>> change that. Those who really care about speed cannot make a >>> balanced >>> decision about Python without considering psyco, pyrex, numpy, and >>> the >>> like as part of the total development environment. >> That's overreaction, I think. People always say this and that about >> python - python is slow, and python is not like java, and python does >> not have static type checks... And what? Should the webmasters adapt >> the >> site to every complain? > > Perhaps a link on the main page to a "for even more speed" page where > we > can combine advice on how to improve application performance (measure, > abstract, choose good data structures) with references to these other > projects for particular applications. This is the place to say things > like "operating on every character of a string is seldom efficient." +1 Many examples of common idioms and tasks and how to do them right in Python would also be useful for such a page. //Simon From allison at sumeru.stanford.EDU Thu Dec 9 21:18:07 2004 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Thu Dec 9 21:18:59 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com> Message-ID: <Pine.LNX.4.10.10412091151530.11555-100000@sumeru.stanford.EDU> The goal here is to make Python better known and to counter some of the prevalent myths. One way to accomplish this goal is to publish literate technical articles with real content including performance measurements and pointers to the code. Perhaps Guido could be a real-life N. Bourbaki and co-publish with developers. A significant issue is: where to publish. Certainly publishing in the technical society journals (ACM & IEEE CS) would make sense. Software Practice and Experience would also be good. Likewise DDJ and Game Developer Magazine. But I suspect that much of our target audience will be missed if we limit ourselves to these journals. We also need to get articles placed in the mass market computer mags. And we need to build an editorial relationship with the mass market journals so that when X says they think "Python is slow" there's some they know they can call for a truth-squad quote. And, parenthetically, I continue to be amazed at the number of projects that use Python, but do it in stealth-mode and view it as their silver-bullet and competative edge. I wish more people would publish their experience. From glyph at divmod.com Thu Dec 9 22:11:33 2004 From: glyph at divmod.com (Glyph Lefkowitz) Date: Thu Dec 9 22:10:35 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> Message-ID: <1102626693.16291.57.camel@localhost> On Wed, 2004-12-08 at 17:39 -0500, Phillip J. Eby wrote: > The only thing that will fix the PR issue is to have a Python compiler > distributed as part of the language. It doesn't matter if it doesn't > support the full generality of Python, or even if it doesn't speed many > operations up much. The only real requirements are that it can be used to > produce "native" executables, and that it be an official part of the > language, not a separately-distributed tool like Psyco or Pyrex. Then, it > will perhaps be a sufficient "security blanket" to stop people FUDding > about it. I am aware that Pyrex is still in flux, so perhaps it is too soon to propose this even for 2.5, but I think it's worth bringing up anyway: I would like Pyrex to be distributed with the Python core. I agree that it should be modified to produce full .exe files and not just .dlls on Windows, but many potential users *are* seriously concerned about efficiency and not just simplifying distribution. The main benefit would be imparting a Pythonesque flavor to optimized code. The thing which experienced Python-ers say is, technically, correct: "If you need speed, rewrite the hot-spots in C.". To someone already steeped in the Python way, this means, "write your application in Python, do your architecture in Python, profile it, and write the 300 lines or so of your 25000 line application that are *really* speed-critical in a faster language". There are a few, very very specialized areas where this approach is not the best one, but unfortunately C programmers do not hear this when you say it. What they hear is, "If speed is important to your application, write it in C, and only write the parts where you need flexibility in Python. They will all be deadly slow. If you notice that they are slow, you are going to have to rewrite them in C anyway." To wit, Python is only interesting to the people who are going to "script" your application. Although writing code in Pyrex ends up being a technically similar solution, politically it is radically different. If the Python-expert's answer were instead, "write the slow parts in Pyrex", then the C programmer's plan changes - instead of writing in C, because they know they are going to need to rewrite all the slow parts anyway, they realize that there is an explicitly Python way to rewrite the slow parts, that may leverage SOME of their existing C knowledge. They learn Python and appreciate Python secure in the knowledge that this effort will not be wasted. Finally, there are real benefits to Pyrex when making the transition between levels in your optimized code. All the common areas for mistakes when interfacing with Python - missing refcounts when getting, changing, or deleting attributes, calling Python callbacks, and using basic Python data structures - are all taken care of so that the dive into C is far less abrupt. The people this PR is geared to are those who would write in C because of efficiency concerns, so they would likely be looking at Pyrex before choosing to start a large project, and the simple code examples would be both familiar enough (hey, that's a 32 bit integer! I remember that!) but also different enough (wow, you can just call a script with "foo()"?) that they would be impressed. From barry at python.org Thu Dec 9 22:20:32 2004 From: barry at python.org (Barry Warsaw) Date: Thu Dec 9 22:20:37 2004 Subject: Freezing Python (was Re: [Python-Dev] 2.4 news reaches interesting places) In-Reply-To: <5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com> References: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> <5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com> Message-ID: <1102627232.16867.47.camel@geddy.wooz.org> On Thu, 2004-12-09 at 10:30, Phillip J. Eby wrote: > ISTR that for a long time, Visual Basic actually did the same thing. A few > magazines mentioned the fact, but nobody really cared. However, if this is > really a concern, bundle Pyrex as well. Both Pyrex and py2exe are > distutils-based, so at that point you have a complete solution, including > the "C" meme as well as the ".exe" meme. As an aside, I wonder if there's interest in collaborating on freeze tools among the various existing solutions here. I've recently needed to look at this and I chose cx_Freeze, primarily because it supported the platforms I needed. It's a very good tool. I've had some conversations with Anthony Tuininga on the cx_Freeze mailing list and there seems to be some interest in perhaps putting together a SIG or something. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041209/ce332aec/attachment.pgp From bob at redivi.com Thu Dec 9 22:30:30 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Dec 9 22:31:08 2004 Subject: Freezing Python (was Re: [Python-Dev] 2.4 news reaches interesting places) In-Reply-To: <1102627232.16867.47.camel@geddy.wooz.org> References: <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208201940.0210b270@mail.telecommunity.com> <5.1.1.6.0.20041209102402.042c8580@mail.telecommunity.com> <1102627232.16867.47.camel@geddy.wooz.org> Message-ID: <8CDE78F4-4A29-11D9-8CCD-000A9567635C@redivi.com> On Dec 9, 2004, at 4:20 PM, Barry Warsaw wrote: > On Thu, 2004-12-09 at 10:30, Phillip J. Eby wrote: > >> ISTR that for a long time, Visual Basic actually did the same thing. >> A few >> magazines mentioned the fact, but nobody really cared. However, if >> this is >> really a concern, bundle Pyrex as well. Both Pyrex and py2exe are >> distutils-based, so at that point you have a complete solution, >> including >> the "C" meme as well as the ".exe" meme. > > As an aside, I wonder if there's interest in collaborating on freeze > tools among the various existing solutions here. I've recently needed > to look at this and I chose cx_Freeze, primarily because it supported > the platforms I needed. It's a very good tool. > > I've had some conversations with Anthony Tuininga on the cx_Freeze > mailing list and there seems to be some interest in perhaps putting > together a SIG or something. That's certainly worth doing. Currently we have, in general use, at least cx_Freeze, py2exe, and py2app. I have been abstracting away a lot of the dependency finding issues in py2app into a cross-platform module called modulegraph <http://svn.red-bean.com/bob/py2app/trunk/src/modulegraph/>. Modulegraph supercedes the standard library modulefinder. Because it uses a graph data structure, it gives you the flexibility say "pydoc doesn't *really* depend on Tkinter". It also has functionality that lets you specify implicit dependencies (via PyImport_.. or such), so that it plays nicely enough with C extensions. This code is basically at the point where py2exe and cx_Freeze could import it and use it instead of modulefinder (with a small amount of hacking, the API is different). -bob From aahz at pythoncraft.com Thu Dec 9 22:35:56 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Dec 9 22:35:58 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <Pine.LNX.4.10.10412091151530.11555-100000@sumeru.stanford.EDU> References: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com> <Pine.LNX.4.10.10412091151530.11555-100000@sumeru.stanford.EDU> Message-ID: <20041209213556.GB4013@panix.com> On Thu, Dec 09, 2004, Dennis Allison wrote: > > And, parenthetically, I continue to be amazed at the number of projects > that use Python, but do it in stealth-mode and view it as their > silver-bullet and competative edge. I wish more people would publish > their experience. http://www.paulgraham.com/avg.html -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From pje at telecommunity.com Thu Dec 9 22:37:59 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Dec 9 22:38:58 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <1102626693.16291.57.camel@localhost> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com> At 04:11 PM 12/9/04 -0500, Glyph Lefkowitz wrote: >On Wed, 2004-12-08 at 17:39 -0500, Phillip J. Eby wrote: > > > The only thing that will fix the PR issue is to have a Python compiler > > distributed as part of the language. It doesn't matter if it doesn't > > support the full generality of Python, or even if it doesn't speed many > > operations up much. The only real requirements are that it can be used to > > produce "native" executables, and that it be an official part of the > > language, not a separately-distributed tool like Psyco or Pyrex. Then, it > > will perhaps be a sufficient "security blanket" to stop people FUDding > > about it. > >I am aware that Pyrex is still in flux, so perhaps it is too soon to >propose this even for 2.5, but I think it's worth bringing up anyway: I >would like Pyrex to be distributed with the Python core. I agree that >it should be modified to produce full .exe files and not just .dlls on >Windows, but many potential users *are* seriously concerned about >efficiency and not just simplifying distribution. +1 on all the stuff you said, with one minor exception. Pyrex-the-language is often unpythonically ugly and verbose at present. If Python had an official syntax for optional static type declaration, Pyrex's syntax could be aligned with that, and that would at least eliminate most of the inline 'cdef' ugliness, leaving only C type declarations and Python property declarations as the main syntax issues to be resolved. (Maybe by using something like the 'ctypes' API, and having the compiler recognize that API, such that C is directly callable from Python anyway, so compiling or interpreting Python makes no difference to ability to access C... but I digress.) Of course, this would take some effort from the core developers, especially Guido, to consider the various syntax needs and formulate official solutions. But if it were done, the Python-vs.-Pyrex distinction could fade away altogether, replaced with the meme, "just add type declarations to slow parts, and tell Python you want the module compiled to C." IOW, if Pyrex is merely the name of a compiler, not a separate language, then our master plan for world domination is complete. :) From michael.walter at gmail.com Thu Dec 9 22:43:19 2004 From: michael.walter at gmail.com (Michael Walter) Date: Thu Dec 9 22:43:22 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <1102626693.16291.57.camel@localhost> <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com> Message-ID: <877e9a17041209134356f9290e@mail.gmail.com> If I parse you correctly, this would be great. - Michael On Thu, 09 Dec 2004 16:37:59 -0500, Phillip J. Eby <pje@telecommunity.com> wrote: > At 04:11 PM 12/9/04 -0500, Glyph Lefkowitz wrote: > > > >On Wed, 2004-12-08 at 17:39 -0500, Phillip J. Eby wrote: > > > > > The only thing that will fix the PR issue is to have a Python compiler > > > distributed as part of the language. It doesn't matter if it doesn't > > > support the full generality of Python, or even if it doesn't speed many > > > operations up much. The only real requirements are that it can be used to > > > produce "native" executables, and that it be an official part of the > > > language, not a separately-distributed tool like Psyco or Pyrex. Then, it > > > will perhaps be a sufficient "security blanket" to stop people FUDding > > > about it. > > > >I am aware that Pyrex is still in flux, so perhaps it is too soon to > >propose this even for 2.5, but I think it's worth bringing up anyway: I > >would like Pyrex to be distributed with the Python core. I agree that > >it should be modified to produce full .exe files and not just .dlls on > >Windows, but many potential users *are* seriously concerned about > >efficiency and not just simplifying distribution. > > +1 on all the stuff you said, with one minor exception. Pyrex-the-language > is often unpythonically ugly and verbose at present. If Python had an > official syntax for optional static type declaration, Pyrex's syntax could > be aligned with that, and that would at least eliminate most of the inline > 'cdef' ugliness, leaving only C type declarations and Python property > declarations as the main syntax issues to be resolved. (Maybe by using > something like the 'ctypes' API, and having the compiler recognize that > API, such that C is directly callable from Python anyway, so compiling or > interpreting Python makes no difference to ability to access C... but I > digress.) > > Of course, this would take some effort from the core developers, especially > Guido, to consider the various syntax needs and formulate official > solutions. But if it were done, the Python-vs.-Pyrex distinction could > fade away altogether, replaced with the meme, "just add type declarations > to slow parts, and tell Python you want the module compiled to C." > > IOW, if Pyrex is merely the name of a compiler, not a separate language, > then our master plan for world domination is complete. :) > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com > From glyph at divmod.com Thu Dec 9 23:11:45 2004 From: glyph at divmod.com (Glyph Lefkowitz) Date: Thu Dec 9 23:10:47 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041209162547.06531700@mail.telecommunity.com> Message-ID: <1102630305.16291.62.camel@localhost> On Thu, 2004-12-09 at 16:37 -0500, Phillip J. Eby wrote: > +1 on all the stuff you said, with one minor exception. Pyrex-the-language > is often unpythonically ugly and verbose at present. Personally I have no problem with Pyrex's current tradeoffs, but given the amount of effort that this is going to inevitably involve, I'm sure that at least some of the changes you suggested would be necessary. So... +0 on your suggestions, I suppose. From faassen at infrae.com Fri Dec 10 01:09:05 2004 From: faassen at infrae.com (Martijn Faassen) Date: Fri Dec 10 01:09:08 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <41B8E921.2090903@infrae.com> Guido van Rossum wrote: [snip] > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? One thing you could do is announce loudly that the European Union is actually funding a international consortium of scientists, industry and software engineers to speed up Python (among other things :) -- PyPy. I'm sure that a press release about this in the right channels could make quite a bit of positive noise. "Python, backed by the EU". Of course one worry is that the 'slow bureaucracy' meme would rub off on it. Of course any press release would need to be discussed with the PyPy people. Regards, Martijn From bob at redivi.com Fri Dec 10 02:17:21 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Dec 10 02:17:58 2004 Subject: [Python-Dev] Proposed Mac OS X change for 2.3.5 and 2.4.1 - MACOSX_DEPLOYMENT_TARGET vs. Python, round two, fight! Message-ID: <3DBFC242-4A49-11D9-8CCD-000A9567635C@redivi.com> I've included a lot of background information here, if you just want to know the details of the proposed change, skip to the bottom. As some of you may know, Python 2.4's configure script and distutils has some tricky behavior with regard to the ``MACOSX_DEPLOYMENT_TARGET`` environment variable on Mac OS X 10.3 and later. Unless otherwise specified, assume that I am talking about a build environment of Mac OS X 10.3 or later. This behavior is as follows: If ``MACOSX_DEPLOYMENT_TARGET`` is not set during configure: 1. distutils builds modules with ``-F/Python/Installation/Location -framework Python`` as per usual 2.3.x behavior if it is also not set 2. If ``MACOSX_DEPLOYMENT_TARGET`` is set during a later run of distutils, then distutils complains that "10.3" mismatches the configure time setting of "" This Python framework has the following misfeatures: 1. All your extensions are dependent on the installation location of this particular Python 2. This installation of Python 2.4 *may break the building of extensions* for any previous version of Python that uses the same distutils behavior. It will certainly break them if they are installed to the same framework directory, but if the version of Python 2.3 is early enough, such as the stock 2.3.0, it will break that too. Also, any future version of Python installed to the same frameworks directory *will break the building of extensions* for this Python installation! 3. The Python framework will not be compatible with versions of Mac OS X earlier than 10.3 anyway due to changes in libSystem! This is stupid, and it should NOT be default behavior! If ``MACOSX_DEPLOYMENT_TARGET`` is set to "10.3" or higher during configure: 1. distutils builds modules with ``-undefined dynamic_lookup`` 2. If ``MACOSX_DEPLOYMENT_TARGET`` is set to something other than "10.3", or unset, it will complain that the current setting mismatches the configure setting of "10.3" The features: 1. All your extensions are independent of the Python installation location, and are thus compatible with any other Python with the same major version. 2. This installation of Python 2.4 will still, unavoidably, break the building of extensions for any previous version of Python using the bad distutils behavior mentioned above. This installation is not susceptible to breakage, however. The misfeatures: 1. You need to have ``MACOSX_DEPLOYMENT_TARGET`` set during configure, most people don't know to do this. 2. You also need to have ``MACOSX_DEPLOYMENT_TARGET`` set when using distutils. Even less people know to do this. The solution to this is to have the following behavior instead: 1. If no ``MACOSX_DEPLOYMENT_TARGET`` is set during configure, and the build machine is Mac OS X 10.3 or later, then set it to "10.3". 2. If no ``MACOSX_DEPLOYMENT_TARGET`` is set during distutils, but it WAS set during configure, then set it to the configure time value. 3. Otherwise, if it is set to a LOWER value, then fail. Checking for an exact match is bad, because the user or extension author should be free to build a particular extension using 10.4 specific features against a Python that is 10.3+ compatible. For a build machine running Mac OS X 10.2 or earlier, it should ignore all of this behavior as per usual. Unless anyone has a reasonable objection to this proposed solution, then we should make sure it goes into Python 2.4.1 and Python 2.3.5. These threads might also be of interest: http://mail.python.org/pipermail/pythonmac-sig/2004-November/012192.html http://mail.python.org/pipermail/pythonmac-sig/2004-December/012237.html http://mail.python.org/pipermail/pythonmac-sig/2004-December/012275.html -bob From skip at pobox.com Thu Dec 9 13:12:31 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Dec 10 03:29:04 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> Message-ID: <16824.16687.343664.151558@montanaro.dyndns.org> Raymond> * Any PR effort should also emphasize that no usability Raymond> trade-offs were made along the way. A number of features Raymond> make Py2.4 easier to use than 1.5.6: list comps, genexps, Raymond> generators, sets, nested scopes, int/long unification, Raymond> decorators, etc. Not to mention which, such beasts aren't commonly available for C. What about C++? I found it interesting that a guy at work wrote a string caching class for our C++ programmers to use. He told me he got the idea from Python's int caching. Skip From anthony at interlink.com.au Fri Dec 10 03:31:20 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Dec 10 03:31:37 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE4A915A@au3010avexu1.global.avaya.com> Message-ID: <200412101331.21291.anthony@interlink.com.au> On Friday 10 December 2004 06:25, Delaney, Timothy C (Timothy) wrote: > Michael Hudson wrote: > > Anthony's Australian, people expect this sort of thing from him :) > > As another Australian, I think that "Making Python Not Suck" implies > that if you don't do extra things, Python Sucks. > > This is not a good thing IMO. > > "Making Python Suck Less" would be even worse. Don't worry - it will have a nice friendly title by the time I present it. I'm currently aiming for OSCON, which isn't until August. It's going to be aimed at people who _know_ Python, but aren't necessarily experts, and focus not just on performance, but maintainable and sane code (for instance, not many people seem to realise that __del__ methods are almost always not the right answer to a problem). > How about "Python - You Can Have Your Cake And Eat It Too". Nah - I have a bunch of ideas kicking around in my head, something will pop out. I doubt I'll ever be able to top the title of my pycon talk, though... Anthony From kdart at kdart.com Fri Dec 10 04:41:19 2004 From: kdart at kdart.com (Keith Dart) Date: Fri Dec 10 04:41:30 2004 Subject: [Python-Dev] PyOS_InputHook enhancement proposal In-Reply-To: <20041209131555.GB10465@panix.com> References: <41B7D8C5.9000008@ims.u-tokyo.ac.jp> <20041209131555.GB10465@panix.com> Message-ID: <41B91ADF.6040800@kdart.com> Aahz wrote: > On Thu, Dec 09, 2004, Michiel Jan Laurens de Hoon wrote: > >>My suggestion is therefore to replace PyOS_InputHook by two functions >>PyOS_AddInputHook and PyOS_RemoveInputHook, and let Python keep track of >>which hooks are installed. This way, an extension module can add a hook >>function without having to worry about other extension modules trying >>to use the same hook. >> >>Any comments? Would I need to submit a PEP for this proposal? > > > Because this is only for the C API, your best bet is to write a patch > and submit it to SF. If people whine or it gets rejected, then write a > PEP. I did modify the readline module that hooks this and can call back to a Python function. There are also methods for installing and removing the Python function. I did this for a different reason. I need Python signal handlers to run, and they don't run when the execution path is currently in some C code (such as readline). The hook forces the interpreter to run, and check for signals as a side effect. Using this I can be sitting in raw_input(), or interactive mode, and signal handlers are still run. If you want it, let me know. Actually, the modded readline module is part of pyNMS, on sourceforge. -- \/ \/ (O O) -- --------------------oOOo~(_)~oOOo---------------------------------------- Keith Dart <kdart@kdart.com> vcard: <http://www.kdart.com/~kdart/kdart.vcf> public key: ID: F3D288E4 URL: <http://www.kdart.com/~kdart/public.key> ============================================================================ From arigo at tunes.org Fri Dec 10 10:41:28 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri Dec 10 10:50:49 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz> References: <ca471dc204120814186f5dc423@mail.gmail.com> <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz> Message-ID: <20041210094128.GA15865@vicky.ecs.soton.ac.uk> Hi Andrew, On Thu, Dec 09, 2004 at 03:32:10PM +1300, Andrew McGregor wrote: > The other thing we can do is finish the portable backend for psyco and > make it a standard module. Then Python won't be slow, it will be > compiled, and py2exe will be able to make a single-file executable. You probably mean that Psyco can dynamically compile Python bytecodes even if they have been hidden into an .exe file by py2exe. At first reading, it appears that you are propagating the misconception "compiler == speed == .exe" i.e. Psyco could "compile your program into an .exe file". As far as I see, this is definitely not possible. Of course, for PR purposes the difference is small -- an .exe file that runs at compiled speed, or at least machine-level VM speed -- but advertising should work around and not build on top of existing confusion IMHO. A bientot, Armin From arigo at tunes.org Fri Dec 10 11:06:59 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri Dec 10 11:16:15 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <001401c4dda9$ad605680$e841fea9@oemcomputer> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> Message-ID: <20041210100659.GB15865@vicky.ecs.soton.ac.uk> Hi, On Wed, Dec 08, 2004 at 11:43:49PM -0500, Raymond Hettinger wrote: > Acceptance for Py2.4 partially hinges on how quickly third party apps > have their binaries updated. > > I wonder if there is anything we can do to help. For people like myself, Linux programmers not developing on Windows every day, there is precious little information available about how to compile our extension modules for the new Windows distribution. I was actually very disappointed to have to google my way around until I found a page that explained to me that to use Microsoft's free compilers you need to manually patch the distutils included with 2.4. (I know this is being worked on right now but I'd have expected it to be present in the 2.4 release.) (The only page I could find still refers to a pre-final 2.4 so their distutils patch doesn't apply without hand-editing, though that should be fixed by now.) Moreover, the standard documentation's "Extending and Embedding" has apparently not been updated at all since 2.3. That's quite unexpected too... In other words, if you want 3rd parties to compile Windows binaries for 2.4, tell them how. (I even hoped that just doing "python setup.py build" would fail but refer me to a web page that explains me which free compilers I should get and install from Microsoft.) A bientot, Armin. From skip at pobox.com Fri Dec 10 11:49:30 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Dec 10 11:49:47 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <20041210094128.GA15865@vicky.ecs.soton.ac.uk> References: <ca471dc204120814186f5dc423@mail.gmail.com> <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz> <20041210094128.GA15865@vicky.ecs.soton.ac.uk> Message-ID: <16825.32570.898575.571826@montanaro.dyndns.org> >> The other thing we can do is finish the portable backend for psyco >> and make it a standard module. Then Python won't be slow, it will be >> compiled, and py2exe will be able to make a single-file executable. Armin> You probably mean that Psyco can dynamically compile Python Armin> bytecodes even if they have been hidden into an .exe file by Armin> py2exe. I didn't read it that way. My impression was that py2exe be modified to include and enable psyco if it's available when building an .exe. You would, in theory, get a single file distribution as well as dynamic compilation. Skip From theller at python.net Fri Dec 10 12:12:36 2004 From: theller at python.net (Thomas Heller) Date: Fri Dec 10 12:11:49 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <16825.32570.898575.571826@montanaro.dyndns.org> (Skip Montanaro's message of "Fri, 10 Dec 2004 04:49:30 -0600") References: <ca471dc204120814186f5dc423@mail.gmail.com> <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz> <20041210094128.GA15865@vicky.ecs.soton.ac.uk> <16825.32570.898575.571826@montanaro.dyndns.org> Message-ID: <y8g676fv.fsf@python.net> Skip Montanaro <skip@pobox.com> writes: > >> The other thing we can do is finish the portable backend for psyco > >> and make it a standard module. Then Python won't be slow, it will be > >> compiled, and py2exe will be able to make a single-file executable. > > Armin> You probably mean that Psyco can dynamically compile Python > Armin> bytecodes even if they have been hidden into an .exe file by > Armin> py2exe. > > I didn't read it that way. My impression was that py2exe be modified to > include and enable psyco if it's available when building an .exe. You > would, in theory, get a single file distribution as well as dynamic > compilation. I haven't tried it, but using psyco in a script and building an exe from it with py2exe should work right out of the box - but maybe this isn't what you had in mind? And single-file exes with py2exe are on my plan for a medium time range, although currently I'm not really sure which way to go. Thomas From mwh at python.net Fri Dec 10 12:56:38 2004 From: mwh at python.net (Michael Hudson) Date: Fri Dec 10 12:56:39 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <e8bf7a5304120911564cef0c53@mail.gmail.com> (Jeremy Hylton's message of "Thu, 9 Dec 2004 14:56:41 -0500") References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <5.1.1.6.0.20041209131843.038f3530@mail.telecommunity.com> <CDB95F16-4A16-11D9-88DA-000A95A50FB2@fuhm.net> <e8bf7a5304120911564cef0c53@mail.gmail.com> Message-ID: <2mfz2epds9.fsf@starship.python.net> Jeremy Hylton <jhylton@gmail.com> writes: > I agree, although it's not clear to me how much faster it will be in > the future. Making a *fast* Python based on our own virtual execution > environment (as opposed to piggybacking a JVM or CLR) is a big > project. It's not clear than anyone has enough resources to make > credible effort I think the EU does! Cheers, mwh -- NUTRIMAT: That drink was individually tailored to meet your personal requirements for nutrition and pleasure. ARTHUR: Ah. So I'm a masochist on a diet am I? -- The Hitch-Hikers Guide to the Galaxy, Episode 9 From mwh at python.net Fri Dec 10 13:02:59 2004 From: mwh at python.net (Michael Hudson) Date: Fri Dec 10 13:03:01 2004 Subject: [Python-Dev] PyOS_InputHook enhancement proposal In-Reply-To: <41B91ADF.6040800@kdart.com> (Keith Dart's message of "Thu, 09 Dec 2004 19:41:19 -0800") References: <41B7D8C5.9000008@ims.u-tokyo.ac.jp> <20041209131555.GB10465@panix.com> <41B91ADF.6040800@kdart.com> Message-ID: <2mbrd2pdho.fsf@starship.python.net> Keith Dart <kdart@kdart.com> writes: > I did modify the readline module that hooks this and can call back > to a Python function. There are also methods for installing and > removing the Python function. I did this for a different reason. I > need Python signal handlers to run, and they don't run when the > execution path is currently in some C code (such as readline). The > hook forces the interpreter to run, and check for signals as a side > effect. Using this I can be sitting in raw_input(), or interactive > mode, and signal handlers are still run. Have you seen what happened to the signal handling code in readline in Python 2.4? I don't think your modifications will be needed anymore, but you should check. Cheers, mwh -- I don't remember any dirty green trousers. -- Ian Jackson, ucam.chat From p.f.moore at gmail.com Fri Dec 10 13:06:01 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Fri Dec 10 13:06:04 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <20041210100659.GB15865@vicky.ecs.soton.ac.uk> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> Message-ID: <79990c6b041210040671584abc@mail.gmail.com> On Fri, 10 Dec 2004 10:06:59 +0000, Armin Rigo <arigo@tunes.org> wrote: > For people like myself, Linux programmers not developing on Windows every day, > there is precious little information available about how to compile our > extension modules for the new Windows distribution. I was actually very > disappointed to have to google my way around until I found a page that > explained to me that to use Microsoft's free compilers you need to manually > patch the distutils included with 2.4. (I know this is being worked on right > now but I'd have expected it to be present in the 2.4 release.) (The only > page I could find still refers to a pre-final 2.4 so their distutils patch > doesn't apply without hand-editing, though that should be fixed by now.) [...] > In other words, if you want 3rd parties to compile Windows binaries for 2.4, > tell them how. I think that the details about the Microsoft free compilers is a bit misleading. Sure, it's great that it exists, but at the present time, it is not the best free option (too many downloads required, too many rough edges). For most C extensions, the best free option is mingw. This is fully supported by distutils, and has been for a while. It is documented in the manuals, but basically all you need is to do the following: 1. Obtain a reasonably recent mingw (3.3.3 definitely works, you need msvcr71 support). 2. Install it, and make sure it is in your PATH. 3. Build libpython24.a as documented in the Python manuals (recent versions of mingw can read MS import libraries, so this may no longer be needed, but I haven't tested this yet). 4. python setup.py build --compiler=mingw32. The only difficulties are: 1. Obtaining support libraries like jpeg, sdl, etc. It *may* be a nuisance getting mingw-compatible build, but as I say above, MS-format libraries may now be fine, and there are good sources of mingw-compatible libraries like gnuwin32.sourceforge.net. 2. Microsoft-specific code. (As I'm replying to Armin, maybe the fact that inline assembler formats are different is relevant :-) :-)) To be honest, I'd only ever use the MS free compilers if I had a critical need to build an extension which had some highly MS-specific code or build processes. And even then, I'd probably give up and wait for someone with MSVC7 to produce a binary... If anyone has any particular modules (of their own, or third party) they have problems with, I'd be happy to have a go at building, and report back. Maybe a page on the Python Wiki about building modules using mingw would be worth adding. Hmm, might do that tonight... Hope this helps, Paul. From skip at pobox.com Fri Dec 10 16:10:41 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Dec 10 16:10:52 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <y8g676fv.fsf@python.net> References: <ca471dc204120814186f5dc423@mail.gmail.com> <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz> <20041210094128.GA15865@vicky.ecs.soton.ac.uk> <16825.32570.898575.571826@montanaro.dyndns.org> <y8g676fv.fsf@python.net> Message-ID: <16825.48241.997055.384959@montanaro.dyndns.org> Thomas> I haven't tried it, but using psyco in a script and building an Thomas> exe from it with py2exe should work right out of the box - but Thomas> maybe this isn't what you had in mind? I was thinking of implicitly mixing in psyco, even if the script didn't use it. Maybe I have too much faith in psyco. ;-) Skip From pje at telecommunity.com Fri Dec 10 16:17:42 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Dec 10 16:19:19 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <20041210100659.GB15865@vicky.ecs.soton.ac.uk> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <001401c4dda9$ad605680$e841fea9@oemcomputer> Message-ID: <5.1.1.6.0.20041210101609.020ed6e0@mail.telecommunity.com> At 10:06 AM 12/10/04 +0000, Armin Rigo wrote: >Hi, > >On Wed, Dec 08, 2004 at 11:43:49PM -0500, Raymond Hettinger wrote: > > Acceptance for Py2.4 partially hinges on how quickly third party apps > > have their binaries updated. > > > > I wonder if there is anything we can do to help. > >For people like myself, Linux programmers not developing on Windows every day, >there is precious little information available about how to compile our >extension modules for the new Windows distribution. I personally find MinGW (by way of Cygwin) to be the easiest way to do it. I posted here previously with the procedure and even a script to prepare the necessary libpython24.a file. From arigo at tunes.org Fri Dec 10 18:07:50 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri Dec 10 18:16:55 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <16825.32570.898575.571826@montanaro.dyndns.org> References: <ca471dc204120814186f5dc423@mail.gmail.com> <86F33E75-498A-11D9-BB31-000D93C603C0@indranet.co.nz> <20041210094128.GA15865@vicky.ecs.soton.ac.uk> <16825.32570.898575.571826@montanaro.dyndns.org> Message-ID: <20041210170750.GA12032@vicky.ecs.soton.ac.uk> Hi Skip, On Fri, Dec 10, 2004 at 04:49:30AM -0600, Skip Montanaro wrote: > > >> The other thing we can do is finish the portable backend for psyco > >> and make it a standard module. Then Python won't be slow, it will be > >> compiled, and py2exe will be able to make a single-file executable. > > Armin> You probably mean that Psyco can dynamically compile Python > Armin> bytecodes even if they have been hidden into an .exe file by > Armin> py2exe. > > I didn't read it that way. My impression was that py2exe be modified to > include and enable psyco if it's available when building an .exe. You > would, in theory, get a single file distribution as well as dynamic > compilation. Yes, I agree with this. What I meant is that when I first read the original paragraph (the 1st one quoted above), I thought it meant that in the future py2exe and Psyco could be combined in such a way that we'd essentially have a compiler from Python producing a "classical" compiled binary. A lot of people could read it that way. The question is if we should advertise a Psyco+py2exe combination using a similar wording, such that it superficially sounds like we are doing a "classical" compilation from .py to .exe, whereas it actually means that we are hiding Psyco with the bytecodes in a .exe. After all, from a user's (or journalist's) point of view the result is similar, performancewise. Whether Psyco is reliable enough for this is yet another issue... I'd classify Psyco as "mostly reliable" only... A bientot, Armin. From arigo at tunes.org Fri Dec 10 18:19:21 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri Dec 10 18:28:22 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <79990c6b041210040671584abc@mail.gmail.com> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> Message-ID: <20041210171921.GB12032@vicky.ecs.soton.ac.uk> Hi, On Fri, Dec 10, 2004 at 12:06:01PM +0000, Paul Moore wrote: > For most C extensions, the best free option is mingw. Sorry, I was not aware that mingw supports the new VC7.1-type of runtime that is needed for the extension module to load with the official Python 2.4 distribution. Note that compiling with Microsoft's free compiler (after the huge downloads and the manual patching) worked just fine, so I think it is worth a note somewhere. Another note: can you report on whether building libpython24.a can be skipped for mingw? I'm thinking about the specific situation where we want on-site compilation of extension modules with a minimal number of things to install first. E.g. if we need to compile libpython24.a it means we need to fetch the Python sources themselves first. (A use case is for prorgams that dynamically produce and compile extension modules, as "weave" or the PyPy test suite do.) A bientot, Armin. From pje at telecommunity.com Fri Dec 10 19:05:17 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Dec 10 19:07:03 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <20041210171921.GB12032@vicky.ecs.soton.ac.uk> References: <79990c6b041210040671584abc@mail.gmail.com> <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> Message-ID: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> At 05:19 PM 12/10/04 +0000, Armin Rigo wrote: >Another note: can you report on whether building libpython24.a can be skipped >for mingw? I'm thinking about the specific situation where we want on-site >compilation of extension modules with a minimal number of things to install >first. E.g. if we need to compile libpython24.a it means we need to fetch the >Python sources themselves first. Actually, no, you don't need the sources. See: http://mail.python.org/pipermail/python-dev/2004-January/041676.html for a script that builds libpython24.a from the python24.lib distributed with Python for Windows. From bob at redivi.com Fri Dec 10 19:12:04 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Dec 10 19:12:42 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> References: <79990c6b041210040671584abc@mail.gmail.com> <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> Message-ID: <FF3971F9-4AD6-11D9-80F2-000A9567635C@redivi.com> On Dec 10, 2004, at 1:05 PM, Phillip J. Eby wrote: > At 05:19 PM 12/10/04 +0000, Armin Rigo wrote: > >> Another note: can you report on whether building libpython24.a can be >> skipped >> for mingw? I'm thinking about the specific situation where we want >> on-site >> compilation of extension modules with a minimal number of things to >> install >> first. E.g. if we need to compile libpython24.a it means we need to >> fetch the >> Python sources themselves first. > > Actually, no, you don't need the sources. See: > > http://mail.python.org/pipermail/python-dev/2004-January/041676.html > > for a script that builds libpython24.a from the python24.lib > distributed with Python for Windows. Shouldn't this be distributed with binary distributions of Python, to save people the trouble? Or, if it can be skipped, the procedure for doing a mingw build with the .lib should be documented and that'd be the end of it. -bob From pje at telecommunity.com Fri Dec 10 19:19:10 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Dec 10 19:20:58 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <FF3971F9-4AD6-11D9-80F2-000A9567635C@redivi.com> References: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> <79990c6b041210040671584abc@mail.gmail.com> <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com> At 01:12 PM 12/10/04 -0500, Bob Ippolito wrote: >On Dec 10, 2004, at 1:05 PM, Phillip J. Eby wrote: >>At 05:19 PM 12/10/04 +0000, Armin Rigo wrote: >> >>>Another note: can you report on whether building libpython24.a can be >>>skipped >>>for mingw? I'm thinking about the specific situation where we want on-site >>>compilation of extension modules with a minimal number of things to install >>>first. E.g. if we need to compile libpython24.a it means we need to >>>fetch the >>>Python sources themselves first. >> >>Actually, no, you don't need the sources. See: >> >>http://mail.python.org/pipermail/python-dev/2004-January/041676.html >> >>for a script that builds libpython24.a from the python24.lib distributed >>with Python for Windows. > >Shouldn't this be distributed with binary distributions of Python, to save >people the trouble? The Python developers who produce the Windows binaries don't use mingw/cygwin, so this would put a maintenance burden on them. >Or, if it can be skipped, the procedure for doing a mingw build with the >.lib should be documented and that'd be the end of it. It's actually documented now, in the "Installing Python Modules" manual. See: http://docs.python.org/inst/tweak-flags.html#SECTION000622000000000000000 It just would need to have the libpython.a instructions removed in that case. From amk at amk.ca Fri Dec 10 20:01:55 2004 From: amk at amk.ca (A.M. Kuchling) Date: Fri Dec 10 20:02:27 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <20041209141104.GA18348@phd.pp.ru> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> <20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org> <20041209141104.GA18348@phd.pp.ru> Message-ID: <20041210190155.GA11443@rogue.amk.ca> On Thu, Dec 09, 2004 at 05:11:04PM +0300, Oleg Broytmann wrote: > some popular areas. Let's add another topic, "Making things fast". Let's > even make it the first topic, though I personnaly dont see a need for > this. The topic guides are migrating into the Wiki, and there's already a Wiki page about this: http://www.python.org/moin/PythonSpeed --amk From phd at mail2.phd.pp.ru Fri Dec 10 20:07:58 2004 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Fri Dec 10 20:08:00 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <20041210190155.GA11443@rogue.amk.ca> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> <20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org> <20041209141104.GA18348@phd.pp.ru> <20041210190155.GA11443@rogue.amk.ca> Message-ID: <20041210190758.GC16678@phd.pp.ru> On Fri, Dec 10, 2004 at 02:01:55PM -0500, A.M. Kuchling wrote: > On Thu, Dec 09, 2004 at 05:11:04PM +0300, Oleg Broytmann wrote: > > some popular areas. Let's add another topic, "Making things fast". Let's > > even make it the first topic, though I personnaly dont see a need for > > this. > > The topic guides are migrating into the Wiki, and there's already a Wiki page > about this: http://www.python.org/moin/PythonSpeed Thank you. Actually, I've thought about wiki when I was writing that email, but I was afarid that people who (in my very humble opinion) are overreacting - will not tolerate putting such an important topic into a wiki. My personnal opinion is that a good topic about the speed of Python in the wiki is enough. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From jhylton at gmail.com Fri Dec 10 20:28:13 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Dec 10 20:28:16 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <20041210190155.GA11443@rogue.amk.ca> References: <ca471dc204120814186f5dc423@mail.gmail.com> <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> <20041209102250.GB12506@phd.pp.ru> <cp9m4m$isn$1@sea.gmane.org> <20041209141104.GA18348@phd.pp.ru> <20041210190155.GA11443@rogue.amk.ca> Message-ID: <e8bf7a530412101128326aea7b@mail.gmail.com> On Fri, 10 Dec 2004 14:01:55 -0500, A.M. Kuchling <amk@amk.ca> wrote: > On Thu, Dec 09, 2004 at 05:11:04PM +0300, Oleg Broytmann wrote: > > some popular areas. Let's add another topic, "Making things fast". Let's > > even make it the first topic, though I personnaly dont see a need for > > this. > > The topic guides are migrating into the Wiki, and there's already a Wiki page > about this: http://www.python.org/moin/PythonSpeed The Wiki entry seems to reinforce the impression that bugged Guido to begin with. It provides a bunch of "but ..." explanations about why Python's speed isn't that important. Python is slow, but "speed of development is far more important." If you want to avoid people saying "Python is slow, but ..." you need to give them a different message. "Writing code quickly is crucial in today's marketplace." (whatever, that's not much of a message.) David's dynamic languages whitepaper has more of message, for example. Jeremy From p.f.moore at gmail.com Fri Dec 10 21:40:10 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Fri Dec 10 21:40:13 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <20041210171921.GB12032@vicky.ecs.soton.ac.uk> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> <20041210171921.GB12032@vicky.ecs.soton.ac.uk> Message-ID: <79990c6b041210124039803de9@mail.gmail.com> On Fri, 10 Dec 2004 17:19:21 +0000, Armin Rigo <arigo@tunes.org> wrote: > Another note: can you report on whether building libpython24.a can be skipped > for mingw? A first attempt seems to almost work. There is a problem with structures, however - I get an error about unresolved references to _imp___Py_NoneStruct. My guess is that there's some issue with resolving references to data (as opposed to functions) from DLLs. I'll investigate further and then report back. But this is specifically if you *don't* build libpytho24.a. If you build that, there's no problem at all. Paul. From p.f.moore at gmail.com Fri Dec 10 23:20:29 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Fri Dec 10 23:20:32 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <79990c6b041210124039803de9@mail.gmail.com> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> <20041210171921.GB12032@vicky.ecs.soton.ac.uk> <79990c6b041210124039803de9@mail.gmail.com> Message-ID: <79990c6b0412101420522d59f3@mail.gmail.com> On Fri, 10 Dec 2004 20:40:10 +0000, Paul Moore <p.f.moore@gmail.com> wrote: > On Fri, 10 Dec 2004 17:19:21 +0000, Armin Rigo <arigo@tunes.org> wrote: > > Another note: can you report on whether building libpython24.a can be skipped > > for mingw? > > A first attempt seems to almost work. There is a problem with > structures, however - I get an error about unresolved references to > _imp___Py_NoneStruct. My guess is that there's some issue with > resolving references to data (as opposed to functions) from DLLs. I'll > investigate further and then report back. OK. After some more extensive investigation: 1. You cannot use python24.lib (ie, skip building libpython24.a). If you do, data values exported from python24.dll (specifically, Py_NoneStruct, which is quite important :-)) do not get exported correctly. I don't know exactly why - I guess it's something to do with how the MS LIB format works. 2. You can, however, link direct to python24.dll, which works fine. Unfortunately, there are 2 problems with this - first, distutils doesn't support it as things stand, and second, it isn't possible to get the full path of the Python DLL without using Mark Hammond's pywin32, or custom C code :-( (You need to get from sys.dllhandle to the filename, and the API to do this isn't exposed). 3. In some cases, an extension built with mingw still contains references to msvcrt (I think Martin has noted this). This seems *not* to be an issue. What is happening, is that any symbols referenced from user code are satisfied from msvcr71, and so match the Python DLL. However, if the mingw startup code uses symbols which have not been satisfied by the time all references from user code are resolved, then these references are satisfied from msvcrt. But I can't see any way that these references are going to do any harm, as they are startup code, which by its nature will have been written to work cleanly in this situation (linking like this is explicitly supported by mingw). Martin, as the CRT compatibility expert, does this sound reasonable? The mingw people certainly seem to think that the situation is OK... You could probably incorporate Philip Eby's script, posted here a while ago, which he mentioned earlier in this thread, into a setup.py so that if libpython24.a didn't already exist, the setup script would build it. This would give you a reasonably clean way of making sure it was present. Philip's script doesn't need anything but mingw installed, AIUI. Longer term, I'll see if I can put together a distutils patch to make it reference pythonXX.dll directly, so removing the need for libpythonXX.a in the future. Obviously, this is only going to be relevant for Python 2.5 and later, though. Sorry for the long mail, but I thought it would be useful to get the details in the archives... Paul. From python at rcn.com Fri Dec 10 23:42:29 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Dec 10 23:45:36 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <e8bf7a530412101128326aea7b@mail.gmail.com> Message-ID: <001d01c4df09$87e5caa0$e841fea9@oemcomputer> > The Wiki entry seems to reinforce the impression that bugged Guido to > begin with. It provides a bunch of "but ..." explanations about why > Python's speed isn't that important. Python is slow, but "speed of > development is far more important." I felt the same way when reading it. Also, it seemed to embody the political outlook that optimization is sinful. The document could be much more positive, fact based, and informative. Also, the wording seems somewhat outdated. A draft for a new entry is included below. Review and feedback are requested. Raymond Techniques for Improving Performance and Scalability ==================================================== Here are some coding guidelines for applications that demand peek performance (in terms of memory utilization, speed, or scalability). Use the best algorithms and fastest tools ----------------------------------------- . Membership testing with sets and dictionaries is much faster, O(1), than searching sequences, O(n). When testing "a in b", b should be a set or dictionary instead of a list or tuple. . String concatenation is best done with ''.join(seq) which is an O(n) process. In contrast, using the '+' or '+=' operators can result in an O(n**2) process because new strings may be built for each intermediate step. Py2.4 mitigates this issue somewhat; however, ''.join(seq) remains the best practice. . Many tools come in both list form and iterator form (map and itertools.imap, list comprehension and generator expressions, dict.items and dict.iteritems). In general, the iterator forms are more memory friendly and more scalable. They are preferred whenever a real list is not required. . Many core building blocks are coded in optimized C. Applications that take advantage of them can make substantial performance gains. The building blocks include all of the builtin datatypes (lists, tuples, sets, and dictionaries) and extension modules like array, itertools, and collections.deque. . Likewise, the builtin functions run faster than hand-built equivalents. For example, map(operator.add, v1, v2) is faster than map(lambda x,y: x+y, v1, v2). . List comprehensions run a bit faster than equivalent for-loops. . Lists perform well as fixed length arrays and as stacks. For queue applications using pop(0) or insert(0,v)), collections.deque() offers superior O(1) performance because it avoids rebuilding a full list for each insertion or deletion, an O(n) process. . Custom sort ordering is best performed with Py2.4's key= option or with the traditional decorate-sort-undecorate technique. Both approaches call the key function just once per element. In contrast, sort's cmp= option is called many times per element during a sort. For example, sort(key=str.lower) is faster than sort(cmp=lambda a,b: cmp(a.lower(), b.lower())). Take advantage of interpreter optimizations ------------------------------------------- . In functions, local variables are accessed more quickly than global variables, builtins, and attribute lookups. So, it is sometimes worth localizing variable access in inner-loops. For example, the code for random.shuffle() localizes access with the line, random=self.random. That saves the shuffling loop from having to repeatedly lookup self.random. Outside of loops, the gain is minimal and rarely worth it. . The previous recommendation is a generalization of the rule to factor constant expressions out of loops. Also, constant folding should be done manually. Inside loops, write "x=3" instead of "x=1+2". . Function call overhead is large compared to other instructions. Accordingly, it is sometimes worth in-lining code in the middle of a few time-critical loops. . Starting with Py2.3, the interpreter optimizes "while 1" to just a single jump. In contrast "while True" takes several more steps. While the latter is preferred for clarity, time critical code should use the first form. . Multiple assignment is slower than individual assignment. For example "x,y=a,b" is slower than "x=a; y=b". However, multiple assignment is faster for variable swaps. For example, "x,y=y,x" is faster than "t=x; x=y; y=t". . Chained comparisons are faster than using the "and" operator. Write "x < y < z" instead of "x < y and y < z". . A few fast approaches should be considered hacks and reserved for only the most demanding applications. For example, "x and True or False" is faster than "bool(x)". Take advantage of diagnostic tools ---------------------------------- . The hotshot and profile modules help identify performance bottlenecks. . The timeit module offers immediate performance comparisons between alternative approaches to writing individual lines of code. Performance can dictate overall strategy ---------------------------------------- . SAX is typically faster and more memory efficient than DOM approaches to XML. . Threading can improve the response time in applications that would otherwise waste time waiting for I/O. . Select can help minimize the overhead for polling multiple sockets. From martin at v.loewis.de Sat Dec 11 00:45:01 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 11 00:45:06 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <20041210100659.GB15865@vicky.ecs.soton.ac.uk> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> Message-ID: <41BA34FD.6050306@v.loewis.de> Armin Rigo wrote: > In other words, if you want 3rd parties to compile Windows binaries for 2.4, > tell them how. The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke python setup.py bdist_wininst That's not too hard to do, I think. Regards, Martin From martin at v.loewis.de Sat Dec 11 00:52:52 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 11 00:52:55 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com> References: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> <79990c6b041210040671584abc@mail.gmail.com> <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com> Message-ID: <41BA36D4.2080903@v.loewis.de> Phillip J. Eby wrote: > The Python developers who produce the Windows binaries don't use > mingw/cygwin, so this would put a maintenance burden on them. If this were completely automatic (e.g. part of msi.py), I'd happily add all utilities needed to integrated this into the build process. For this to happen: 1. Somebody else needs to do the actual work. I.e. write a patch to msi.py that invokes all the necessary magic, and test that patch. 2. The patch should avoid manual configuration of the "use cygwin" case. I.e. it should try to find the cygwin on the path, or in their standard location, or use any other mechanism available to locate cygwin. It should produce an error message if cygwin tools cannot be found, and then proceed without out creating libpython24.a Regards, Martin From chui.tey at advdata.com.au Fri Dec 10 12:53:16 2004 From: chui.tey at advdata.com.au (Chui G. Tey) Date: Sat Dec 11 04:24:58 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) Message-ID: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> >Raymond: > >Acceptance for Py2.4 partially hinges on how quickly third party apps >have their binaries updated. > >I wonder if there is anything we can do to help. One good way of helping out is to provide an dynamic loading function that third party modules could access the basic python functions such as PyArgParseTuple, PyString_AsString etc regardless of which python the user is running. This would be similar to the COM approach. You can load all the function pointers into a struct and then call them. Third party modules would link against this DLL independent of which python is being used. Chui From gvwilson at cs.toronto.edu Tue Dec 7 21:32:58 2004 From: gvwilson at cs.toronto.edu (Greg Wilson) Date: Sat Dec 11 04:25:52 2004 Subject: [Python-Dev] decorators article for DDJ? Message-ID: <Pine.GSO.4.58.0412071532260.9235@dvp.cs> Hi folks. Is anyone interested in doing an article on Python decorators for "Doctor Dobb's Journal"? If so, please drop me a line... Thanks, Greg From neal at metaslash.com Thu Dec 9 03:33:50 2004 From: neal at metaslash.com (Neal Norwitz) Date: Sat Dec 11 04:26:49 2004 Subject: [Python-Dev] removing aclocal.m4 Message-ID: <20041209023350.GB7217@janus.swcomplete.com> Can we remove aclocal.m4? The last log message states: fix for bug #811160 - autoconf vs. hp/ux system header files. also applied to release23-maint. Note that aclocal.m4 can go away when autoconf 2.58 is out. And configure says: # Generated by GNU Autoconf 2.59 for python 2.5. Neal From ncoghlan at iinet.net.au Sat Dec 11 05:41:24 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Dec 11 05:41:31 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <001d01c4df09$87e5caa0$e841fea9@oemcomputer> References: <001d01c4df09$87e5caa0$e841fea9@oemcomputer> Message-ID: <41BA7A74.1010302@iinet.net.au> Raymond Hettinger wrote: > guidelines for applications that demand peek performance (in terms of memory Peak performance, perhaps? :) Anyway, it looks pretty good to me, but I have a few additional ideas. Add a section of Caveats (we know they exist - might as well be upfront about it): Caveats -------- . This document is based primarily on the CPython interpreter from www.python.org. However, most of the recommendations should apply to any Python interpreter that supports the mentioned feature. . For small data sets, some of the suggestions will perform more slowly than the approaches they are given as alternatives too. This is due to the fact that those suggestions introduce a little more fixed overhead in order to avoid overhead in processing each data item. The crossover point (where the more scaleable approach begins to show a net performance gain) is generally application specific. Use the diagnostic tools mentioned later to make an informed decision for your application. > Use the best algorithms and fastest tools > ----------------------------------------- > . Membership testing with sets and dictionaries is much faster, > O(1), than searching sequences, O(n). When testing "a in b", b should > be a set or dictionary instead of a list or tuple. Should the bisect module be mentioned? If you have a sorted list already, using bisect may be faster than constructing an intermediate set. Also, you can use bisect if an item's *position* in the list is important. list.index(x) uses a linear search (since it can't assume a sorted list), whereas bisect expects the list to be pre-sorted. > intermediate step. Py2.4 mitigates this issue somewhat; however, > ''.join(seq) remains the best practice. I'd say 'The CPython 2.4 interpreter', rather than just Py2.4. > . List comprehensions run a bit faster than equivalent for-loops. I'd move this to the next section (the algorithms are the same, but the interpreter can speed up one more than the other) > . Custom sort ordering is best performed with Py2.4's key= option or with the > traditional decorate-sort-undecorate technique. Both approaches call the key > function just once per element. In contrast, sort's cmp= option is called > many times per element during a sort. For example, sort(key=str.lower) is > faster than sort(cmp=lambda a,b: cmp(a.lower(), b.lower())). If sorting is going to be performed repeatedly (e.g. maintaining a list in sorted order), it may be feasible to store the list of keys, rather than regenerating them every time. This also plays nicely with using the bisect module to update the list (as the list of keys is available to determine the correct index for insertion). I can't find a 'sorted list' recipe in the Cookbook, so I might have to create one. > Take advantage of interpreter optimizations > ------------------------------------------- [snip] . Prefer iteration (using for loops, list comprehensions, or generator expressions) over iterables to explicit invocation of .next() > Take advantage of diagnostic tools > ---------------------------------- > > . The hotshot and profile modules help identify performance bottlenecks. Currently profile is a fair bit more useful than hotshot, mainly due to the fact that it isolates time spent in C functions from the Python code that calls those functions. Perhaps another section for External Libraries? If you're doing serious number crunching in Python, using NumPy is practically a requirement. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From mdehoon at ims.u-tokyo.ac.jp Sat Dec 11 07:13:39 2004 From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon) Date: Sat Dec 11 07:10:24 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> References: <79990c6b041210040671584abc@mail.gmail.com> <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> Message-ID: <41BA9013.50101@ims.u-tokyo.ac.jp> >> E.g. if we need to compile libpython24.a it means we need to >> fetch the Python sources themselves first. > > Actually, no, you don't need the sources. See: > > http://mail.python.org/pipermail/python-dev/2004-January/041676.html > > for a script that builds libpython24.a from the python24.lib distributed > with Python for Windows. > Previously I built libpython24.a; it can be downloaded from my website. See http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/cygwin.html --Michiel. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon From martin at v.loewis.de Sat Dec 11 09:39:28 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 11 09:39:31 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) In-Reply-To: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> Message-ID: <41BAB240.6060101@v.loewis.de> Chui G. Tey wrote: > One good way of helping out is to provide an dynamic loading function > that third party modules could access the basic python functions such as > PyArgParseTuple, PyString_AsString etc regardless of which python the > user is running. This would be similar to the COM approach. You can load > all the function pointers into a struct and then call them. > > Third party modules would link against this DLL independent of which > python is being used. I believe this is not implementable: How can the DLL know which Python DLL to use? Regards, Martin From martin at v.loewis.de Sat Dec 11 09:53:29 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 11 09:53:32 2004 Subject: [Python-Dev] removing aclocal.m4 In-Reply-To: <20041209023350.GB7217@janus.swcomplete.com> References: <20041209023350.GB7217@janus.swcomplete.com> Message-ID: <41BAB589.4030303@v.loewis.de> Neal Norwitz wrote: > Can we remove aclocal.m4? The last log message states: > > fix for bug #811160 - autoconf vs. hp/ux system header files. > also applied to release23-maint. > > Note that aclocal.m4 can go away when autoconf 2.58 is out. It appears to me that 2.59 indeed fixes the HP-UX problem; a diff with and without aclocal.m4 shows chunks like cat confdefs.h >>conftest.$ac_ext cat >>conftest.$ac_ext <<_ACEOF /* end confdefs.h. */ +/* Define $ac_func to an innocuous variant, in case <limits.h> declares $ac_func. + For example, HP-UX 11i <limits.h> declares gettimeofday. */ +#define $ac_func innocuous_$ac_func + /* System header to define __stub macros and hopefully few prototypes, which can conflict with char $ac_func (); below. Prefer <limits.h> to <assert.h> if __STDC__ is defined, since - <limits.h> exists even on freestanding compilers. Under hpux, - including <limits.h> includes <sys/time.h> and causes problems - checking for functions defined therein. */ -#if defined (__STDC__) && !defined (_HPUX_SOURCE) + <limits.h> exists even on freestanding compilers. */ + +#ifdef __STDC__ # include <limits.h> #else # include <assert.h> #endif + +#undef $ac_func + So they manage to get limits.h define, say, innoucuous_clock(), instead of clock(), whereas we currently avoid including limits.h on HP-UX. IOW, it seems like it should work, but somebody should test it on HP-UX to be sure. Regards, Martin From bob at redivi.com Sat Dec 11 10:51:30 2004 From: bob at redivi.com (Bob Ippolito) Date: Sat Dec 11 10:52:07 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) In-Reply-To: <41BAB240.6060101@v.loewis.de> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> Message-ID: <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com> On Dec 11, 2004, at 3:39 AM, Martin v. L?wis wrote: > Chui G. Tey wrote: >> One good way of helping out is to provide an dynamic loading function >> that third party modules could access the basic python functions such >> as >> PyArgParseTuple, PyString_AsString etc regardless of which python the >> user is running. This would be similar to the COM approach. You can >> load >> all the function pointers into a struct and then call them. Third >> party modules would link against this DLL independent of which >> python is being used. > > I believe this is not implementable: How can the DLL know which Python > DLL to use? Well for py2app on Mac OS X, I wrote an executable stub that chooses a Python runtime from an XML file, looks up and binds a few symbols from it dynamically, and then starts doing stuff. Works for 2.3 (as long as it's not compiled debug or trace refs) and 2.4 (universally, because it has exported functions for incref and decref). For extension modules, this makes much less sense, but I wanted to be able to bootstrap Python programs with a precompiled executable that I didn't have to maintain across several versions. -bob From fredrik at pythonware.com Sat Dec 11 11:14:38 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat Dec 11 11:12:23 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <cpeh62$ldh$1@sea.gmane.org> > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. fwiw, IDG's Computer Sweden, "sweden's leading IT-newspaper" has a surprisingly big Python article in their most recent issue: PYTHON FEELS WELL Better performance biggest news in 2.4 and briefly interviews swedish zope-developer Johan Carlsson and Python- Ware co-founder Håkan Karlsson. </F> PS. the heading is a play on the swedish phrase "må pyton", which means "feel like crap"... From fredrik at pythonware.com Sat Dec 11 15:06:05 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat Dec 11 15:03:51 2004 Subject: [Python-Dev] Re: Supporting Third Party Modules (was The other Py2.4issue) References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> Message-ID: <cpeuo1$g58$1@sea.gmane.org> Martin v. Löwis wrote: >> Third party modules would link against this DLL independent of which >> python is being used. > > I believe this is not implementable: How can the DLL know which Python > DLL to use? the Python interpreter could initialize this DLL, using some suitable mechanism. alternatively, it could figure out what EXE you used to start the program in the first place, and figure out what Python DLL that EXE uses. my ExeMaker tool can do that, for example: http://effbot.org/zone/exemaker.htm (see the #!python.exe entry in the table) </F> From faassen at infrae.com Sat Dec 11 15:07:27 2004 From: faassen at infrae.com (Martijn Faassen) Date: Sat Dec 11 15:07:30 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <cpeh62$ldh$1@sea.gmane.org> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> Message-ID: <41BAFF1F.3060200@infrae.com> Fredrik Lundh wrote: [snip] > fwiw, IDG's Computer Sweden, "sweden's leading IT-newspaper" has a > surprisingly big Python article in their most recent issue: > > PYTHON FEELS WELL > Better performance biggest news in 2.4 [snip] Perhaps the message getting out is actually that Python's performance is getting better, so nothing to worry about. The original article that Guido cited had a headline along the same lines: "Faster Python grabs programmers". So perhaps there really is nothing to worry about in this department: the headlines are saying "Python now faster". That to me sounds like an excellent way to combat "Python is slow" memes. :) Regards, Martijn From gvanrossum at gmail.com Sat Dec 11 17:22:17 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat Dec 11 17:22:20 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <cpeh62$ldh$1@sea.gmane.org> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> Message-ID: <ca471dc204121108225511d43@mail.gmail.com> > fwiw, IDG's Computer Sweden, "sweden's leading IT-newspaper" has a > surprisingly big Python article in their most recent issue: > > PYTHON FEELS WELL > Better performance biggest news in 2.4 > > and briefly interviews swedish zope-developer Johan Carlsson and Python- > Ware co-founder H?kan Karlsson. Maybe we've done this to ourselves then. I'm sure the interviewer didn't know what was the biggest news until after Johan and H?kan told him. And this despite all the functional improvements that are much more interesting for most Python programmers: generator expressions, decorators, a ton of new modules... Ah, but several of those modules are rewrites in C of modules that were previously written in Python. Maybe we really *should* stop emphasizing speed so much to ourselves, and focus more on fixing bugs and adding features instead of looking for speedup opportunities. I think Python's message ought to be more something like "programming for the rest of us -- a child can learn Python, but Python's no toy." BTW I strongly disagree that making easy .EXE binaries available will address this issue; while not bundled, there are plenty of solutions for maning .EXEs for those who need them, and this is not something that typically worries managers. But the perception of Python as "slow" does worry managers. (Wish I could add more to this thread but my family is calling...) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From phd at mail2.phd.pp.ru Sat Dec 11 17:27:14 2004 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Sat Dec 11 17:27:16 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <ca471dc204121108225511d43@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> Message-ID: <20041211162714.GA32590@phd.pp.ru> On Sat, Dec 11, 2004 at 08:22:17AM -0800, Guido van Rossum wrote: > But the perception of Python as > "slow" does worry managers. Much more those managers are worried that Python isn't backed by a large corporation. For Java there is Sun, for Visual Basic there is the biggest and most powerful corporation. But who is for Python? Who is responsible? Whom to blame? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From arigo at tunes.org Sat Dec 11 17:55:09 2004 From: arigo at tunes.org (Armin Rigo) Date: Sat Dec 11 18:04:11 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <41BA34FD.6050306@v.loewis.de> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> Message-ID: <20041211165508.GA10252@vicky.ecs.soton.ac.uk> Hi Martin, On Sat, Dec 11, 2004 at 12:45:01AM +0100, "Martin v. L?wis" wrote: > The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke > > python setup.py bdist_wininst > > That's not too hard to do, I think. Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest time and money on Windows tools just to compile my extension module for Windows users... Armin From arigo at tunes.org Sat Dec 11 17:55:16 2004 From: arigo at tunes.org (Armin Rigo) Date: Sat Dec 11 18:04:22 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com> References: <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> <79990c6b041210040671584abc@mail.gmail.com> <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <79990c6b041210040671584abc@mail.gmail.com> <5.1.1.6.0.20041210125911.0283dec0@mail.telecommunity.com> <5.1.1.6.0.20041210131425.030fd360@mail.telecommunity.com> Message-ID: <20041211165516.GB10252@vicky.ecs.soton.ac.uk> Hi, On Fri, Dec 10, 2004 at 01:19:10PM -0500, Phillip J. Eby wrote: > >>http://mail.python.org/pipermail/python-dev/2004-January/041676.html > > > >Shouldn't this be distributed with binary distributions of Python, to save > >people the trouble? > > The Python developers who produce the Windows binaries don't use > mingw/cygwin, so this would put a maintenance burden on them. This thread started as "how to convince people to build their extension modules quickly for Python 2.4". I was just suggesting that taking a bit of that burden away from us non-Windows-users would be helpful :-) > >Or, if it can be skipped, the procedure for doing a mingw build with the > >.lib should be documented and that'd be the end of it. > > It's actually documented now, in the "Installing Python Modules" manual. Oh right... Well, I didn't notice because, oddly, there is another set of pages on the same subject (but with different details and up-to-date-ness) in "Extending and Embedding the Python Interpreter". I guess that one of them should be removed and point to the other... A bientot, Armin. From pje at telecommunity.com Sat Dec 11 18:07:16 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Dec 11 18:05:43 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <ca471dc204121108225511d43@mail.gmail.com> References: <cpeh62$ldh$1@sea.gmane.org> <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> Message-ID: <5.1.1.6.0.20041211120241.03c12b50@mail.telecommunity.com> At 08:22 AM 12/11/04 -0800, Guido van Rossum wrote: >BTW I strongly disagree that making easy .EXE binaries available will >address this issue; while not bundled, there are plenty of solutions >for maning .EXEs for those who need them, and this is not something >that typically worries managers. But the perception of Python as >"slow" does worry managers. The relevant memes are that "compiled == fast", and that ".exe == compiled". It's not about .exe as a distribution format, but rather that producing an .exe is supporting evidence of having a "real" compiler. IOW, if there exists a compiler, but it doesn't produce .exe's, it won't be considered a "real" compiler by many programmers. From fredrik at pythonware.com Sat Dec 11 19:06:19 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat Dec 11 19:04:06 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places References: <ca471dc204120814186f5dc423@mail.gmail.com><cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> Message-ID: <cpfcqh$ejc$1@sea.gmane.org> Guido van Rossum wrote: > > fwiw, IDG's Computer Sweden, "sweden's leading IT-newspaper" has a > > surprisingly big Python article in their most recent issue: > > > > PYTHON FEELS WELL > > Better performance biggest news in 2.4 > > (hmm. I seem to have accidentally deleted a line here...) > > and briefly interviews swedish zope-developer Johan Carlsson and Python- > > Ware co-founder Håkan Karlsson. > > Maybe we've done this to ourselves then. I'm sure the interviewer > didn't know what was the biggest news until after Johan and Håkan told > him. And this despite all the functional improvements that are much > more interesting for most Python programmers: generator expressions, > decorators, a ton of new modules... I don't know Johan, but I'm pretty sure Håkan didn't talk about performance; he's more interested in *development* speed... (and I don't think he's been following 2.4 that closely). Looking again, the article says: "Among the news in 2.4 is better performance, especially for new functions that were added in 2.3." and then mentions integer/long integer unification and the decimal type as notable additions, and continues "and so it continues, with a long list of improved details". The article also points to www.python.org/2.4, where the first link is: http://www.python.org/2.4/highlights.html which says: Here are the (subjective) highlights of what's new in Python 2.4. * Faster A number of modules that were added in Python 2.3 (such as sets and heapq) have been recoded in C. In addition, there's been a number of other speedups to the interpreter. (See section 8.1, Optimizations, of the "What's New" document for more [this lists a couple of thousand tweaks by Raymond, and Armin's string concatenation hack]). * New language features /.../ unification of integers and long integers /.../ decimal - a new numeric type that allows for the accurate representation of floating point numbers /.../ so I don't think you can blame Johan or Håkan... the writer simply read the python.org material, and picked a couple of things that he found interesting (decorators and generator expressions may be a big thing for an experienced pythoneer, but they are probably a bit too obscure for a general audience...) > Ah, but several of those modules are rewrites in C of modules that > were previously written in Python. Maybe we really *should* stop > emphasizing speed so much to ourselves, and focus more on fixing bugs > and adding features instead of looking for speedup opportunities. yes please. > (Wish I could add more to this thread but my family is calling...) same here. more later. </F> From tismer at stackless.com Sat Dec 11 19:57:55 2004 From: tismer at stackless.com (Christian Tismer) Date: Sat Dec 11 19:57:57 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <20041211165508.GA10252@vicky.ecs.soton.ac.uk> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> <20041211165508.GA10252@vicky.ecs.soton.ac.uk> Message-ID: <41BB4333.2070003@stackless.com> Armin Rigo wrote: > Hi Martin, > > On Sat, Dec 11, 2004 at 12:45:01AM +0100, "Martin v. L?wis" wrote: > >>The straight-forward answer is: Get VC7.1 (aka VS.NET 2003), and invoke >> >>python setup.py bdist_wininst >> >>That's not too hard to do, I think. > > > Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest > time and money on Windows tools just to compile my extension module for > Windows users... Maybe we can set this up as a service? I have the compiler and could do something like doing a checkout, build, and upload new binary for a number of projects. What is needed is a simple but secure notification method. Probably I need one windows machine which is always online. ciao - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From ncoghlan at iinet.net.au Sun Dec 12 01:39:13 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sun Dec 12 01:39:17 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <000201c4df6b$5c9cef60$e841fea9@oemcomputer> References: <000201c4df6b$5c9cef60$e841fea9@oemcomputer> Message-ID: <41BB9331.2060408@iinet.net.au> Raymond Hettinger wrote: <some sensible things> Makes sense to me - I look forward to seeing the next version. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Sun Dec 12 03:20:01 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sun Dec 12 03:20:07 2004 Subject: [Python-Dev] PEP 338: Executing modules inside packages with '-m' Message-ID: <41BBAAD1.4090208@iinet.net.au> Python 2.4's -m command line switch only works for modules directly on sys.path. Trying to use it with modules inside packages will fail with a "Module not found" error. This PEP aims to fix that for Python 2.5. Previously, posting of a draft version of the PEP to python-dev and python-list didn't actually generate any responses. I'm not sure if that's an indication that people don't see the restriction to top-level modules as a problem (and hence think the PEP is unecessary), or think the extension to handle packages is obvious (and hence see no need to comment). For c.l.p, it could just be a sign that Python 2.4 hasn't been out long enough for anyone to care what I'm yabbering on about :) Anyway, all comments are appreciated (even a simple "Sounds good to me"). Cheers, Nick. *********************************************************************** PEP: 338 Title: Executing modules inside packages with '-m' Version: $Revision: 1.2 $ Last-Modified: $Date: 2004/12/11 20:31:10 $ Author: Nick Coghlan <ncoghlan@email.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 16-Oct-2004 Python-Version: 2.5 Post-History: 8-Nov-2004 Abstract ======== This PEP defines semantics for executing modules inside packages as scripts with the ``-m`` command line switch. The proposed semantics are that the containing package be imported prior to execution of the script. Rationale ========= Python 2.4 adds the command line switch ``-m`` to allow modules to be located using the Python module namespace for execution as scripts. The motivating examples were standard library modules such as ``pdb`` and ``profile``. A number of users and developers have requested extension of the feature to also support running modules located inside packages. One example provided is pychecker's ``pychecker.checker`` module. This capability was left out of the Python 2.4 implementation because the appropriate semantics were not entirely clear. The opinion on python-dev was that it was better to postpone the extension to Python 2.5, and go through the PEP process to help make sure we got it right. Scope of this proposal ========================== In Python 2.4, a module located using ``-m`` is executed just as if its filename had been provided on the command line. The goal of this PEP is to get as close as possible to making that statement also hold true for modules inside packages. Prior discussions suggest it should be noted that this PEP is **not** about any of the following: - changing the idiom for making Python modules also useful as scripts (see PEP 299 [1]_). - lifting the restriction of ``-m`` to modules of type PY_SOURCE or PY_COMPILED (i.e. ``.py``, ``.pyc``, ``.pyo``, ``.pyw``). - addressing the problem of ``-m`` not understanding zip imports or Python's sys.metapath. The issues listed above are considered orthogonal to the specific feature addressed by this PEP. Current Behaviour ================= Before describing the new semantics, it's worth covering the existing semantics for Python 2.4 (as they are currently defined only by the source code). When ``-m`` is used on the command line, it immediately terminates the option list (like ``-c``). The argument is interpreted as the name of a top-level Python module (i.e. one which can be found on ``sys.path``). If the module is found, and is of type ``PY_SOURCE`` or ``PY_COMPILED``, then the command line is effectively reinterpreted from ``python <options> -m <module> <args>`` to ``python <options> <filename> <args>``. This includes setting ``sys.argv[0]`` correctly (some scripts rely on this - Python's own ``regrtest.py`` is one example). If the module is not found, or is not of the correct type, an error is printed. Proposed Semantics ================== The semantics proposed are fairly simple: if ``-m`` is used to execute a module inside a package as a script, then the containing package is imported before executing the module in accordance with the semantics for a top-level module. This is necessary due to the way Python's import machinery locates modules inside packages. A package may modify its own __path__ variable during initialisation. In addition, paths may affected by ``*.pth`` files. Accordingly, the only way for Python to reliably locate the module is by importing the containing package and inspecting its __path__ variable. Note that the package is *not* imported into the ``__main__`` module's namespace. The effects of these semantics that will be visible to the executed module are: - the containing package will be in sys.modules - any external effects of the package initialisation (e.g. installed import hooks, loggers, atexit handlers, etc.) Reference Implementation ======================== A reference implementation is available on SourceForge [2]_. In this implementation, if the ``-m`` switch fails to locate the requested module at the top level, it effectively reinterprets the command from ``python -m <script>`` to ``python -m execmodule <script>``. (There is one caveat: when reinterpreted in this way, ``sys.argv[0]`` may not actually contain the filename of ``execmodule``. This only affects ``execmodule`` itself, not the requested module). ``execmodule`` is a proposed standard library module that contains a single function (also called ``execmodule``). When invoked as a script, this module finds and executes the module supplied as the first argument. It adjusts ``sys.argv`` by deleting ``sys.argv[0]`` and replacing the new ``sys.argv[0]`` with the module's filename instead of its Python name. The function ``execmodule`` is like ``execfile``, but uses the Python module namespace to locate the script instead of the filesystem. It has an additional optional argument ``set_argv0`` which causes the filename of the located module to be written to ``sys.argv[0]`` before the module is executed. A hybrid C/Python implementation is used as the Python module is much more flexible and extensible than the equivalent C code would be. It also allows the ``execmodule`` function to be made available. Scripts which execute other scripts (e.g. ``profile``, ``pdb``) have the option to use this function to provide ``-m`` style support for identifying the script to be executed. The Python code for ``execmodule`` has also been posted as a cookbook recipe for Python 2.4 [3]_. Open Issues =========== - choosing a name for the standard library module containing ``execmodule``. The reference implementation uses ``execmodule``. An alternative name proposed on python-dev is ``runpy``. Alternatives ============ The main alternative implementation considered ignored packages' __path__ variables, and looked only in the main package directory. A Python script with this behaviour can be found in the discussion of the ``execmodule`` cookbook recipe [3]_. This approach was not used as it does not meet the main goal of the ``-m`` switch -- to allow the full Python namespace to be used to locate modules for execution. References ========== .. [1] Special __main__() function in modules (http://www.python.org/peps/pep-0299.html) .. [2] Native ``-m`` execmodule support (http://sourceforge.net/tracker/?func=detail&aid=1043356&group_id=5470&atid=305470 ) .. [3] execmodule Python Cookbook Recipe (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From bac at OCF.Berkeley.EDU Sun Dec 12 04:10:09 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Dec 12 04:10:16 2004 Subject: [Python-Dev] python-dev Summary for 2004-10-16 through 2004-10-31 [draft] Message-ID: <41BBB691.40604@ocf.berkeley.edu> Winter Break is upon me which means I have time to catch up on the Summaries. I will definitely be caught up by the end of the month. As for this summary, I will send this out around Wednesday. As always corrections are appreciated. If you feel one of the skipped threads deserves coverage please feel free to write up a summary and I will edit it and include it. -------------------------------------- ===================== Summary Announcements ===================== `Python 2.4`_ final is now out! As I mentioned in the last summary my school schedule this past quarter has been insane. But now I am on Winter Break and will hopefully be able to catch up on my Summary backlog. .. _Python 2.4: http://www.python.org/2.4/ ========= Summaries ========= -------------------------------------------------------- Specifying main functions and calling packages with '-m' -------------------------------------------------------- In my opinion, the new '-m' command line option in Python 2.4 is really handy. But wouldn't it be even handier if you could execute modules in a package? That exact question came up. The reason this kind of thing didn't just go directly into 2.4 was that the semantics are not as obvious nor is it as simple. A PEP is on the way, though. Until then, you can use Nick Coghlan's recipe at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772. This also brought up the discussion of being able to specify a 'main' function to take the place of the good old ``if __name__ == "__main__"`` idiom. Some liked the idea of allowing one to define a function named 'main', others '__main__'. But the discussion never went any farther. This will require a PEP to ever even be seriously considered. Contributing threads: - `python-dev Summary for 2004-09-16 through 2004-09-30 [draft] <>`__ - `Magic main functions <>`__ - `Supporting packages on the command line <>`__ ---------------------------- ConfigParser shootout begins ---------------------------- As mentioned in the `last summary`_, a desire for a replacement for ConfigParser has come into being. It seems the ideas are being hashed out in the wiki at http://www.python.org/moin/ConfigParserShootout . Contributing threads: - `ConfigParser shootout, preliminary entry <>`__ - `What are the goals for ConfigParser 3000? <>`__ ---------------------------------------------------- Making pymalloc friendlier to long running processes ---------------------------------------------------- Pymalloc, when a small chunk of memory is requested that is less than 256 bytes, fetches the memory from a pool of previously used memory that is now available. While this speeds up memory allocation since it does not directly involve calling the OS to free up the memory, it does cause issues for long running processes that have at some point requested a large portion of memory. The example in the OP for this whole topic was an app that needs 1GB for about five minutes, but the amount of allocated memory stays at 1 GB since pymalloc does not free it to the OS. This was brought up back in June and is summarized at http://www.python.org/dev/summary/2004-06-01_2004-06-15.html#id17 . No code has been checked in at the moment to change the behavior partially thanks to the difficulty of the problem. Contributing threads: - `Changing pymalloc behaviour for long running processes <>`__ - `Re: Python-Dev Digest, Vol 15, Issue 46 <>`__ ---------------------------- How to get a patch looked at ---------------------------- Often times people have a specific patch that they want reviewed/applied to solve a specific problem they are having. Unfortunately the Python developers have limited time; we just can't get to every patch there in a timely fashion. This can be a problem who *really* need to get a patch in. Enter Martin v. L?wis and his deal to review a specific patch: 1. Review 10 other patches on SF 2. Send an email to python-dev listing the 10 patches that you reviewed and the one patch you want to be reviewed 3. Your specific patch gets reviewed by Martin You can see the exact details of Martin's requirements at http://mail.python.org/pipermail/python-dev/2004-October/049495.html and can also read http://www.python.org/dev/dev_intro.html for ideas on what to do for reviewing. Contributing threads: - `Patches <>`__ ---------------------------------------------- Discussion of including pysqlite in the stdlib ---------------------------------------------- The idea of including pysqlite_ in the stdlib came up once again (not this is the wrapper for SQLite_ and not SQLite itself). The arguments for including the module were the SQLite is most likely used more than Sleepycat and thus deserved a place in the stdlib if bsddb did. It also goes along with Python's idea of "batteries included". Arguments against started with the idea of sanctioning pysqlite over other SQLite wrappers did not seem proper. People also thought that including bsddb might not be correct anymore and thus should not be used as a basis for including pysqlite. This all then led into a discussion about package management and how to simplify extending what can be installed at compile-time. The idea of pushing PyPI_ came up as well as revising `PEP 206`_. .. _pysqlite: http://pysqlite.org/ .. _SQLite: http://www.sqlite.org/ .. _PyPI: http://www.python.org/pypi .. _PEP 206: http://www.python.org/peps/pep-0206.html Contributing threads: - `SQLite module for Python 2.5 <>`__ =============== Skipped Threads =============== - Python 2.4b1 on win32 installer bug? - test_subprocess 2.4b1 fails on FreeBSD 5.2 - adding Py{String|Unicode}_{Lower|Upper} and fixing CreateProcess in _subprocess.pyd and PyWin32 - IPv6 bug in 2.3.4?? - logging needs better documentation - bsddb todo for someone - auto-upgrade db format - Tests broken on Windows - import w/options - Adding a developer People are only added as a developer if they ask for it and have shown they are going to stick around - audiotest.au and possible copyright issues? - Re: [Python-checkins] python/dist/src/Pythoncompile.c, 2.330, 2.331 - Bug [ 959379 ] Implicit close() should check for errors - weakref callback vs gc vs threads From bac at OCF.Berkeley.EDU Sun Dec 12 04:17:28 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Dec 12 04:17:51 2004 Subject: [Python-Dev] PEP 338: Executing modules inside packages with '-m' In-Reply-To: <41BBAAD1.4090208@iinet.net.au> References: <41BBAAD1.4090208@iinet.net.au> Message-ID: <41BBB848.4040408@ocf.berkeley.edu> Nick Coghlan wrote: > Python 2.4's -m command line switch only works for modules directly on > sys.path. > Trying to use it with modules inside packages will fail with a "Module not > found" error. This PEP aims to fix that for Python 2.5. > > Previously, posting of a draft version of the PEP to python-dev and > python-list > didn't actually generate any responses. I'm not sure if that's an > indication > that people don't see the restriction to top-level modules as a problem > (and > hence think the PEP is unecessary), or think the extension to handle > packages is > obvious (and hence see no need to comment). > > For c.l.p, it could just be a sign that Python 2.4 hasn't been out long > enough for anyone to care what I'm yabbering on about :) > > Anyway, all comments are appreciated (even a simple "Sounds good to me"). > Sounds good to me. =) +1 -Brett From ncoghlan at iinet.net.au Sun Dec 12 05:10:59 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sun Dec 12 05:11:04 2004 Subject: [Python-Dev] python-dev Summary for 2004-10-16 through 2004-10-31 [draft] In-Reply-To: <41BBB691.40604@ocf.berkeley.edu> References: <41BBB691.40604@ocf.berkeley.edu> Message-ID: <41BBC4D3.1050604@iinet.net.au> Brett C. wrote: > This also brought up the discussion of being able to specify a 'main' > function to take the place of the good old ``if __name__ == "__main__"`` > idiom. Some liked the idea of allowing one to define a function named > 'main', others '__main__'. But the discussion never went any farther. > This will require a PEP to ever even be seriously considered. There's a PEP already - PEP 299. The PEP actually describes a reasonable approach, since code using the current idiom is unlikely to define a __main__() function. However, it seems more like a Py3K idea, since if it's only in 2.5 and later, we'd see code like this to support earlier 2.x versions: ========================== def __main__(*args): ... if __name__ == "__main__": import sys as _sys if _sys.version_info < (2, 5): __main__(_sys.argv) ========================== Or, instead (using only the current idiom): ========================== def _main(*args): ... if __name__ == "__main__": import sys as _sys _main(_sys.argv) ========================== So, to my mind, the backwards compatibility issue completely defeats the PEP's goal of finding a cleaner idiom than the current one. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From faassen at infrae.com Sun Dec 12 13:15:29 2004 From: faassen at infrae.com (Martijn Faassen) Date: Sun Dec 12 13:15:32 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041211120241.03c12b50@mail.telecommunity.com> References: <cpeh62$ldh$1@sea.gmane.org> <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <5.1.1.6.0.20041211120241.03c12b50@mail.telecommunity.com> Message-ID: <41BC3661.4030008@infrae.com> Phillip J. Eby wrote: > At 08:22 AM 12/11/04 -0800, Guido van Rossum wrote: > >> BTW I strongly disagree that making easy .EXE binaries available will >> address this issue; while not bundled, there are plenty of solutions >> for maning .EXEs for those who need them, and this is not something >> that typically worries managers. But the perception of Python as >> "slow" does worry managers. > > The relevant memes are that "compiled == fast", and that ".exe == > compiled". > > It's not about .exe as a distribution format, but rather that producing > an .exe is supporting evidence of having a "real" compiler. > > IOW, if there exists a compiler, but it doesn't produce .exe's, it won't > be considered a "real" compiler by many programmers. This is indeed an extremely common set of memes. It especially haunts people who have done some programming in the past but don't really have it as their main focus now. Many managers would be in this group. It's a PC-platform thing mostly. In the early days of the PC, you had BASIC, and you had compiled languages. Interpreted BASIC was considered to be unprofessional and slow. Distributing your program as a .bas file was considered to be a sign of amateurism. Besides, interpreters on that hardware *were* sometimes unacceptably slow. If you got a compiled .exe it was a sign of performance and more professionalism. Microsoft was quite aware of this meme: they did the trick with Visual Basic (packaging their interpreter in the .exe and called it 'compiled'. The "compiled == fast" meme is also very common among programmers themselves; I know I myself had to wrestle free of it (I didn't care about .exes by that time as I was on linux). That's probably why we have so many Python programmers saying, "well, yeah, Python is not as fast as compiled languages but it's fast enough"; that's the counter meme that replaced the "compilation is good" meme that was there in those people before. It's quite possible that some of these programmers influence managers. Regards, Martijn From jjl at pobox.com Sun Dec 12 16:09:57 2004 From: jjl at pobox.com (John J Lee) Date: Sun Dec 12 16:13:03 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> References: <000b01c4dda2$fe7a87e0$e841fea9@oemcomputer> Message-ID: <Pine.LNX.4.58.0412121505030.6267@alice> On Wed, 8 Dec 2004, Raymond Hettinger wrote: > > One thing that bugs me: the article says 3 or 4 times that Python is > > slow, each time with a refutation ("but it's so flexible", "but it's > > fast enough") but still, they sure seem to harp on the point. This is > > a PR issue that Python needs to fight -- any ideas? [...] > * Have python.org prominently feature an article of Python's use in > high-performance environments. IIRC, somebody wrote a realtime voice > over internet system and found that with good design, there was no speed > issue. Also, the cellphone apps may make a good example. [...] IIRC, Alex Martelli mentioned a video application at last year's ACCU / Python UK conference. He said the system never went into production, but it sounded like a good meme from the speed point of view: it triggered surprised and gleeful noises from the mixed Python / C++ / Java audience ;-). John From jjl at pobox.com Sun Dec 12 16:26:00 2004 From: jjl at pobox.com (John J Lee) Date: Sun Dec 12 16:29:06 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> Message-ID: <Pine.LNX.4.58.0412121514220.6267@alice> On Wed, 8 Dec 2004, Phillip J. Eby wrote: > At 02:18 PM 12/8/04 -0800, Guido van Rossum wrote: > >I was pleasantly surprised to find a pointer to this article in a news > >digest that the ACM emails me regularly (ACM TechNews). > > > >http://gcn.com/vol1_no1/daily-updates/28026-1.html > > > >One thing that bugs me: the article says 3 or 4 times that Python is > >slow, each time with a refutation ("but it's so flexible", "but it's > >fast enough") but still, they sure seem to harp on the point. This is > >a PR issue that Python needs to fight -- any ideas? > > The only thing that will fix the PR issue is to have a Python compiler > distributed as part of the language. It doesn't matter if it doesn't I suspect you're correct, but the suggestion elsewhere to bundle py2exe seems likely to be counterproductive to me: merely emphasizing the "interpreterness" of Python the moment the idea spreads that Python-built .exe's are so big because they're just an interpreter plus a script. I'm sure PyPy, if successful, will be a big win on both PR and technical fronts. On a seperate PR issue, I use the word 'script' above advisedly: At work, I've noticed technical employees of clients who use Java often seem to take some satisfaction in referring to our web applications (which of course, consist of who knows how many packages, modules and classes) as "CGI scripts". We do use CGI, but the CGI scripts themselves are always about five lines long and just contain boilerplate code and configuration to kick off our framework. You can see them imagining a great long script named doeverything.cgi... John From fredrik at pythonware.com Sun Dec 12 16:38:43 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun Dec 12 16:38:59 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places References: <5.1.1.6.0.20041208172852.03020bd0@mail.telecommunity.com> <Pine.LNX.4.58.0412121514220.6267@alice> Message-ID: <cphom3$aj2$1@sea.gmane.org> John J Lee wrote: > the idea spreads that Python-built .exe's are so big because they're just > an interpreter plus a script. that's why exemaker is so great, of course: all EXE files are 4 kilobytes. as for the PYC files, hide them in a ZIP archive named DAT, HLP, OCX, or even DLL, and you're done. </F> From martin at v.loewis.de Sun Dec 12 16:40:22 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 16:40:26 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <20041211165508.GA10252@vicky.ecs.soton.ac.uk> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> <20041211165508.GA10252@vicky.ecs.soton.ac.uk> Message-ID: <41BC6666.8000500@v.loewis.de> Armin Rigo wrote: > Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest > time and money on Windows tools just to compile my extension module for > Windows users... If you don't have Windows at all, you cannot create Windows installers for your users, anyway. If you do have Windows, but you normally don't use it for software development (so you have no copy of MSVC), you have to invest money, indeed. You don't have to invest time - atleast not as much time as you need to invest finding out how these alternative compilers work. However, for free software, things work quite differently, anyway: if you (as a package author) don't have the time and money to create a Windows package, tell your users that you won't create one. Instead, ask for a volunteer to create a package for you. If your package has a large Windows user community, somebody will have the money, and they will find the time if the build process is as simple as bdist_wininst. If none of your users volunteers to do the build for you, I would stop worrying about the Windows users. Regards, Martin From martin at v.loewis.de Sun Dec 12 16:45:02 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 16:45:08 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <41BB4333.2070003@stackless.com> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> <20041211165508.GA10252@vicky.ecs.soton.ac.uk> <41BB4333.2070003@stackless.com> Message-ID: <41BC677E.6060604@v.loewis.de> Christian Tismer wrote: > Maybe we can set this up as a service? > I have the compiler and could do something like > doing a checkout, build, and upload new binary > for a number of projects. I don't think this needs to be too automated. Offering this service is a good thing, and if somebody volunteers, I will applaud. However, I believe that it will be used less often than this thread might suggest: Many developers do have VS.NET available, or know somebody in their user community who does (or would know if they asked), so a "global" compilation service would likely be an infrequently-used fallback. Nevertheless: if somebody wants to offer this for, say, 6 months, I'd be really curious what the feedback will be. We could announce this on pydotorg and elsewhere if that volunteers desires. Regards, Martin From martin at v.loewis.de Sun Dec 12 17:02:57 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 17:02:59 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) In-Reply-To: <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com> Message-ID: <41BC6BB1.9010401@v.loewis.de> Bob Ippolito wrote: >> I believe this is not implementable: How can the DLL know which Python >> DLL to use? > > > Well for py2app on Mac OS X, I wrote an executable stub that chooses a > Python runtime from an XML file, looks up and binds a few symbols from > it dynamically, and then starts doing stuff. While that would work, I think this is inappropriate for this specific issue: we want to write extension modules which are independent of the Python version, and might even be used with multiple Python installations on the same system. In that case, adding configuration files won't work, as each usage of the extension might require a different Python DLL. Regards, Martin From erik at heneryd.com Sun Dec 12 17:10:58 2004 From: erik at heneryd.com (Erik Heneryd) Date: Sun Dec 12 17:11:04 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <cpfcqh$ejc$1@sea.gmane.org> References: <ca471dc204120814186f5dc423@mail.gmail.com><cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> Message-ID: <41BC6D92.5050802@heneryd.com> Fredrik Lundh wrote: >>>fwiw, IDG's Computer Sweden, "sweden's leading IT-newspaper" has a >>>surprisingly big Python article in their most recent issue: >>> >>> PYTHON FEELS WELL >>> Better performance biggest news in 2.4 >>> >>>and briefly interviews swedish zope-developer Johan Carlsson and Python- >>>Ware co-founder H?kan Karlsson. >> ... > > so I don't think you can blame Johan or H?kan... the writer simply read the > python.org material, and picked a couple of things that he found interesting > (decorators and generator expressions may be a big thing for an experienced > pythoneer, but they are probably a bit too obscure for a general audience...) I'm a bit puzzled by the last paragraph, where Python is grouped together with PHP and Perl - names starting with p, being popular on Linux and not having big, commercial backers. The article then concludes "Since Python is copyrighted, it's not truly open. However, it can be freely used and redistributed, even commercially." Huh? Where did THAT come from? You might argue the merits of Python being associated with Perl/PHP, but it's a fact that it is. But when it is, it's seen as less free? Erik From martin at v.loewis.de Sun Dec 12 17:19:24 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 17:19:27 2004 Subject: [Python-Dev] Re: Supporting Third Party Modules (was The other Py2.4issue) In-Reply-To: <cpeuo1$g58$1@sea.gmane.org> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> <cpeuo1$g58$1@sea.gmane.org> Message-ID: <41BC6F8C.2030301@v.loewis.de> Fredrik Lundh wrote: > the Python interpreter could initialize this DLL, using some suitable mechanism. What suitable mechanism could that be? Suppose the Python script imports foo.pyd, which in turn is linked with pythonany.dll, which in turn is supposed to find the right Python DLL. Now, python24.dll runs LoadLibary for foo.pyd, but has no control over pythonany.dll. How could the suitable mechanism work? > alternatively, it could figure out what EXE you used to start the program in the > first place, and figure out what Python DLL that EXE uses. my ExeMaker tool > can do that, for example: > > http://effbot.org/zone/exemaker.htm > > (see the #!python.exe entry in the table) That requires a module handle for the EXE, right? How do you get that handle? Out of curiosity: How come that the hModule is the start address of the image? On what Windows versions does that work? Is that documented? Regards, Martin From p.f.moore at gmail.com Sun Dec 12 17:57:07 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Sun Dec 12 17:57:10 2004 Subject: [Python-Dev] PEP 338: Executing modules inside packages with '-m' In-Reply-To: <41BBB848.4040408@ocf.berkeley.edu> References: <41BBAAD1.4090208@iinet.net.au> <41BBB848.4040408@ocf.berkeley.edu> Message-ID: <79990c6b0412120857196eadb0@mail.gmail.com> On Sat, 11 Dec 2004 19:17:28 -0800, Brett C. <bac@ocf.berkeley.edu> wrote: > Nick Coghlan wrote: > > > Anyway, all comments are appreciated (even a simple "Sounds good to me"). > > > > Sounds good to me. =) And me :-) +1 Paul. From carribeiro at gmail.com Sun Dec 12 18:14:09 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Sun Dec 12 18:14:13 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <41BC6D92.5050802@heneryd.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> Message-ID: <864d370904121209147b713f31@mail.gmail.com> On Sun, 12 Dec 2004 17:10:58 +0100, Erik Heneryd <erik@heneryd.com> wrote: > Fredrik Lundh wrote: > >>>fwiw, IDG's Computer Sweden, "sweden's leading IT-newspaper" has a > >>>surprisingly big Python article in their most recent issue: > >>> > >>> PYTHON FEELS WELL > >>> Better performance biggest news in 2.4 > >>> > > >>>and briefly interviews swedish zope-developer Johan Carlsson and Python- > >>>Ware co-founder H?kan Karlsson. > >> > > ... > > > > > so I don't think you can blame Johan or H?kan... the writer simply read the > > python.org material, and picked a couple of things that he found interesting > > (decorators and generator expressions may be a big thing for an experienced > > pythoneer, but they are probably a bit too obscure for a general audience...) > > I'm a bit puzzled by the last paragraph, where Python is grouped > together with PHP and Perl - names starting with p, being popular on > Linux and not having big, commercial backers. The article then > concludes "Since Python is copyrighted, it's not truly open. However, > it can be freely used and redistributed, even commercially." > > Huh? Where did THAT come from? You might argue the merits of Python > being associated with Perl/PHP, but it's a fact that it is. But when it > is, it's seen as less free? The author was probably referring to the old (and as AFAIK already solved) CRNI copyright issue that ocurred into the 1.x to 2.x series transition. It's amazing how old memes from Python keep being remembered and repeated, even years after the fact. It also illustrates something very important - the community is not doing a good job at spreading the news; perhaps we talk too much between ourselves, and too little with the outside market. IMHO the website is a great part of this, its message being more important to "sell" Python than the standard library or native .exes. About the website, a note from my own experience: when I search for documentation on Python, I'm usually directed to some of the mirror of the main python.org site. To find it inside the main site, I have to use "site:python.org", or even "site:docs.python.org". Usually Google does a good job at ranking pages, and if it doesn't rank the main Python website very highly, it's because they're not being referred to. A campaign to ask people to put links back to the canonical documentation at the Python website would be nice. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From p.f.moore at gmail.com Sun Dec 12 18:26:49 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Sun Dec 12 18:26:52 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <41BB4333.2070003@stackless.com> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> <20041211165508.GA10252@vicky.ecs.soton.ac.uk> <41BB4333.2070003@stackless.com> Message-ID: <79990c6b0412120926647402e7@mail.gmail.com> On Sat, 11 Dec 2004 19:57:55 +0100, Christian Tismer <tismer@stackless.com> wrote: > Armin Rigo wrote: > > Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest > > time and money on Windows tools just to compile my extension module for > > Windows users... First of all, I'm assuming this is a timing issue. If I understood your initial posting, your concern was that people wanted Windows build of your extension *now*, and it was going to take you time to make it available. That's a different issue from you not having the facilities to build the Windows installers at all, where you rely on 3rd parties making builds available. As Martin points out, ultimately the provision of Windows binaries is an issue for the extension project - is the demand high enough to justify the effort, can you find tools and/or a helper, etc etc. But the former issue (how quickly you can provide binaries, assuming that you will do so ultimately) is more relevant for python-dev. Specifically, because lack of binary extensions can be a barrier to take-up of the new version. Certainly, in the past, you could pretty much guarantee that there would be very few Windows users testing beta releases, because pywin32 binaries weren't available. With 2.4, I have at least one system I'd upgrade *now* but for the lack of a critical extension in binary form (I haven't yet had the time to try to adapt the build process to mingw for myself). > Maybe we can set this up as a service? That sounds like a good idea. What I'd suggest is needed is a website where the following can take place: 1. People have a way of posting rquests for particular modules. 2. Installers can be uploaded to satisfy those requests. 3. There is somewhere to put step-by-step "build it yourself" instructions, using free components, so that people *without* access to VS.NET can make progress themselves. Obviously, if a particular extension can't be built with free compilers, then binaries or access to VS.NET are the only options. The installers should be clearly noted as having no warranty or support implied, to encourage people to offer binaries they have built without feeling that they are taking on a support burden. Conversely, as soon as "official" binaries are available from the extension project, the binaries available here should be removed (and replaced with a link to the official site) to reinforce the "unofficial" nature of the binaries provided here. The biggest potential issue with such a site is clearly validation. I've no idea how to make something like this work without it being a major virus risk. Which may, sadly, be enough to kill the idea :-( Paul. From carribeiro at gmail.com Sun Dec 12 18:32:03 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Sun Dec 12 18:32:05 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <864d370904121209147b713f31@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> Message-ID: <864d3709041212093274becde2@mail.gmail.com> Hello all, Just to complement my previous remarks, I would like to point out how do a competing language defines itself in its own website. The perl.org website has a simple faq that is a good piece of marketing. What follows are direct quotes, just to point out how ot handle the market perception about their quality & speed. -- "Perl takes the best features from other languages, such as C, awk, sed, sh, and BASIC, among others." (A claim can't possibly be any more generic than this. Strangely enough, it mentions only older languages -- not Java, C++, or even Python (!). This is possibly a sign of an old quote, but anyway: it has a good marketing effect, specially for non-techies.) -- "Perl can be embedded into web servers to speed up processing by as much as 2000%." (BTW, this quote is embarrassing misleading -- it probably means that Perl is 20x slower when started on a request basis by the web server, and that embedding it will accelerate response by a huge factor. I'm sure non-techies will read it as "Perl is able to accelerate my server 20x!") Of course, the point here is not Perl-bashing. The point here is that we should be able to "sell" Python better than we do now, even without the need to resort to such poor measures. I'm sure the Python community does have good & creative people that can write a good "selling" FAQ for Python, emphasizing the main points of the language. For those who believe that a non-profit project should not do any marketing, a reminder. If the perception about Python is one of a slow language, it's much more difficult to find places where you can use Python. In the long run, many of us may be forced to work with other languages & tools, just because that's where the money is. I personally take it a matter of personal interest, because I know how hard it is to "sell" Python to companies here in Brazil. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From bob at redivi.com Sun Dec 12 18:46:18 2004 From: bob at redivi.com (Bob Ippolito) Date: Sun Dec 12 18:46:26 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) In-Reply-To: <41BC6BB1.9010401@v.loewis.de> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com> <41BC6BB1.9010401@v.loewis.de> Message-ID: <BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com> On Dec 12, 2004, at 11:02 AM, Martin v. L?wis wrote: > Bob Ippolito wrote: >>> I believe this is not implementable: How can the DLL know which >>> Python >>> DLL to use? >> Well for py2app on Mac OS X, I wrote an executable stub that chooses >> a Python runtime from an XML file, looks up and binds a few symbols >> from it dynamically, and then starts doing stuff. > > While that would work, I think this is inappropriate for this specific > issue: we want to write extension modules which are independent of > the Python version, and might even be used with multiple Python > installations on the same system. In that case, adding configuration > files won't work, as each usage of the extension might require a > different Python DLL. Yes, of course, I was talking about the executable, not extensions. On Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows extensions to link to no Python whatsoever. The extensions will just find the symbols it needs from some other image already loaded into the process at runtime. If it weren't for the "forced" ABI incompatibility, we'd already have extensions that work cross-Python-major-version (assuming they used a safe subset of functions and structures). -bob From carribeiro at gmail.com Sun Dec 12 18:53:10 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Sun Dec 12 18:53:12 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <41BC6666.8000500@v.loewis.de> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> <20041211165508.GA10252@vicky.ecs.soton.ac.uk> <41BC6666.8000500@v.loewis.de> Message-ID: <864d37090412120953187d31c9@mail.gmail.com> On Sun, 12 Dec 2004 16:40:22 +0100, Martin v. L?wis <martin@v.loewis.de> wrote: > If none of your users volunteers to do the build for you, I would stop > worrying about the Windows users. Sorry, Martin. I understand your point, but I think you are not being realistic. I for myself took the decision to use only free tools for my own development, but I still have to suport my Windows customers. I can't force them to change to Linux. I don't own a copy of MSVC. Also, one of the reasons to choose a third part module is to save time. The moment I am required to recompile everything myself I'm losing this convenience. This of course impacts my ability to focus on my own work, and so the story goes. I'm not saying that module authors should work for free just to save me some time & hassle. It's fair if an author decides to release a Linux-only module. But again -- this is not realistic. The world has a lot of Windows users, and I depend on them for my own income. If I can't find a good set of Python tools for my projects, what should I do? Picking another language is not a choice, mind you :-) All in all, I sincerely hope that this discussion end up in a high note. I'm not qualified to talk about the particulars of C compilers & development environments, but as a Python user, I have hope that a good solution will be found to make the process of building Python extensions for Windows more convenient. The dream scenario is not to require recompiling, at least inside the same major release (2.4 to 2.5, for example). That would be really great. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From fredrik at pythonware.com Sun Dec 12 18:53:27 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun Dec 12 18:53:35 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places References: <e8bf7a530412101128326aea7b@mail.gmail.com> <001d01c4df09$87e5caa0$e841fea9@oemcomputer> Message-ID: <cpi0io$s64$1@sea.gmane.org> > . Multiple assignment is slower than individual assignment. For > example "x,y=a,b" is slower than "x=a; y=b". However, multiple > assignment is faster for variable swaps. For example, "x,y=y,x" is > faster than "t=x; x=y; y=t". marginally faster in 2.4, a lot slower in earlier versions. maybe you should mark sections that rely on 2.4-specific optimizations, so that people who use earlier versions don't end up slowing their programs down... </F> From martin at v.loewis.de Sun Dec 12 19:06:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 19:06:47 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <864d37090412120953187d31c9@mail.gmail.com> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> <20041211165508.GA10252@vicky.ecs.soton.ac.uk> <41BC6666.8000500@v.loewis.de> <864d37090412120953187d31c9@mail.gmail.com> Message-ID: <41BC88B4.2010703@v.loewis.de> Carlos Ribeiro wrote: >>If none of your users volunteers to do the build for you, I would stop >>worrying about the Windows users. > > > Sorry, Martin. I understand your point, but I think you are not being > realistic. I for myself took the decision to use only free tools for > my own development, but I still have to suport my Windows customers. I > can't force them to change to Linux. I don't own a copy of MSVC. Also, > one of the reasons to choose a third part module is to save time. The > moment I am required to recompile everything myself I'm losing this > convenience. This of course impacts my ability to focus on my own > work, and so the story goes. I did not suggest that *all* your Windows users should recompile your module - just a single one would be sufficient. > I'm not saying that module authors should work for free just to save > me some time & hassle. It's fair if an author decides to release a > Linux-only module. But again -- this is not realistic. The world has a > lot of Windows users, and I depend on them for my own income. If I > can't find a good set of Python tools for my projects, what should I > do? Picking another language is not a choice, mind you :-) As I said: Find a volunteer that has the necessary build infrastructure, and have that volunteer build the extension for you. > The dream scenario is not to > require recompiling, at least inside the same major release (2.4 to > 2.5, for example). That would be really great. That is guaranteed. Extensions built for 2.4 will certainly continue to work in 2.4.1, and later 2.4.x. They will stop working with 2.5 (as they are linked with python24.dll). Regards, Martin From martin at v.loewis.de Sun Dec 12 19:09:05 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 19:09:08 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) In-Reply-To: <BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com> <41BC6BB1.9010401@v.loewis.de> <BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com> Message-ID: <41BC8941.7090605@v.loewis.de> Bob Ippolito wrote: > Yes, of course, I was talking about the executable, not extensions. On > Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows > extensions to link to no Python whatsoever. It's the same on SysV ELF shared libraries, and in most other unices. > The extensions will just > find the symbols it needs from some other image already loaded into the > process at runtime. If it weren't for the "forced" ABI incompatibility, > we'd already have extensions that work cross-Python-major-version > (assuming they used a safe subset of functions and structures). Are you talking about a forced ABI incompatibility, beyond the Windows issue of linking with a specific pythonxy.dll? On Unix, you certainly can have extensions across Python major versions. Regards, Martin From nick at craig-wood.com Sun Dec 12 19:15:37 2004 From: nick at craig-wood.com (Nick Craig-Wood) Date: Sun Dec 12 19:15:41 2004 Subject: [Python-Dev] The other Py2.4 issue In-Reply-To: <41BC6666.8000500@v.loewis.de> References: <001401c4dda9$ad605680$e841fea9@oemcomputer> <20041210100659.GB15865@vicky.ecs.soton.ac.uk> <41BA34FD.6050306@v.loewis.de> <20041211165508.GA10252@vicky.ecs.soton.ac.uk> <41BC6666.8000500@v.loewis.de> Message-ID: <20041212181537.GA22022@craig-wood.com> On Sun, Dec 12, 2004 at 04:40:22PM +0100, "Martin v. L?wis" wrote: > Armin Rigo wrote: > >Hum, this is getting into a Linux-vs-Windows argument. I don't > >want to invest time and money on Windows tools just to compile my > >extension module for Windows users... > > If you don't have Windows at all, you cannot create Windows installers > for your users, anyway. Actually you are wrong there! We cross compile our major application (about 500k lines of C/C++) on linux/x86 for various architectures, Windows/x86 being one of them. We create the Windows installer using the Nullsoft installer (which we run under Wine) and compile with a gcc / mingw cross compiler. This all works very well and means we can build all our architectures every night on a single machine. I'm following this thread with interest because we are considering embedding Python into this project, and I'm wondering whether we can cross compile python using mingw (almost certainly by the sound of it), but probably harder would be to make python module build and install system work cross-wise. -- Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick From bob at redivi.com Sun Dec 12 19:24:15 2004 From: bob at redivi.com (Bob Ippolito) Date: Sun Dec 12 19:24:21 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) In-Reply-To: <41BC8941.7090605@v.loewis.de> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com> <41BC6BB1.9010401@v.loewis.de> <BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com> <41BC8941.7090605@v.loewis.de> Message-ID: <07BA42EC-4C6B-11D9-BE18-000A9567635C@redivi.com> On Dec 12, 2004, at 1:09 PM, Martin v. L?wis wrote: > Bob Ippolito wrote: >> Yes, of course, I was talking about the executable, not extensions. >> On Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows >> extensions to link to no Python whatsoever. > > It's the same on SysV ELF shared libraries, and in most other unices. > >> The extensions will just find the symbols it needs from some other >> image already loaded into the process at runtime. If it weren't for >> the "forced" ABI incompatibility, we'd already have extensions that >> work cross-Python-major-version (assuming they used a safe subset of >> functions and structures). > > Are you talking about a forced ABI incompatibility, beyond the Windows > issue of linking with a specific pythonxy.dll? > > On Unix, you certainly can have extensions across Python major > versions. I was talking about PYTHON_API_VERSION, hence "forced" in quotes. Nobody likes to see an ugly error message. -bob From abkhd at hotmail.com Sun Dec 12 20:21:11 2004 From: abkhd at hotmail.com (A.B., Khalid) Date: Sun Dec 12 20:23:05 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue Message-ID: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl> Nick Craig-Wood wrote: >I'm following this thread with interest because we are considering >embedding Python into this project, and I'm wondering whether we can >cross compile python using mingw (almost certainly by the sound of >it), but probably harder would be to make python module build and >install system work cross-wise. I am following this thread with interest too, because I can certainly report that it is now indeed possible and quite easy to compile not only Python's 2.4 extensions but also Python itself in MinGW with the help of pyMinGW[1]. Indeed there is no need to buy or even download 400MB+ of code to do so. After patching the Python 2.4 source with pyMinGW-24, and issuing the command: make -f python24.mak all We get Python and all its subprojects (socket, ssl, etc.) ready to run: $ /c/temp/py24/python/dist/src/MinGW/python -i Python 2.4.1a0 (#60, Dec 6 2004, 21:05:41) [GCC 3.4.1 (mingw special)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>>import sys >>>sys.winver '2.4' >>> To compile Armin's Psyco 1.3 in my MinGW compiled Python, I simply issued: python setup.py build and then python setup.py bdist_wininst And I had a /dist/psyco-1.3.win32-py2.4.exe file ready to be installed. It works fine at this end (note that I poked myself in the Windows registry to insert the pythoncore version information etc., but that should not be the case with those running the official Windows distribution). I only wonder if extensions created this way will work just as good in the latter. Because if they did, then going down the road of extensive downloads or expensive investment would be truly and totally unnecessary. Regards, Khalid [1] pyMinGW can be found here: http://jove.prohosting.com/iwave/ipython/pyMinGW.html _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ From martin at v.loewis.de Sun Dec 12 23:01:01 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 23:01:05 2004 Subject: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) In-Reply-To: <07BA42EC-4C6B-11D9-BE18-000A9567635C@redivi.com> References: <D35EB085E189EB45A1A0535ABEC71D3208762A@adiex2k.advdata.com.au> <41BAB240.6060101@v.loewis.de> <3BBC89A1-4B5A-11D9-80F2-000A9567635C@redivi.com> <41BC6BB1.9010401@v.loewis.de> <BA266FBA-4C65-11D9-BE18-000A9567635C@redivi.com> <41BC8941.7090605@v.loewis.de> <07BA42EC-4C6B-11D9-BE18-000A9567635C@redivi.com> Message-ID: <41BCBF9D.6060900@v.loewis.de> Bob Ippolito wrote: > I was talking about PYTHON_API_VERSION, hence "forced" in quotes. > Nobody likes to see an ugly error message. Ah, that... If PYTHON_API_VERSION changes, it is because there is a real incompatibility, somewhere. It is not automatically bumped with each release. So it is safe to ignore the ugly error message only if you have not used any of the API that became incompatible. Regards, Martin From martin at v.loewis.de Sun Dec 12 23:07:18 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 12 23:07:21 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl> References: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl> Message-ID: <41BCC116.8040202@v.loewis.de> A.B., Khalid wrote: > I am following this thread with interest too, because I can certainly > report that it is now indeed possible and quite easy to compile not only > Python's 2.4 extensions but also Python itself in MinGW with the help of > pyMinGW[1]. Indeed there is no need to buy or even download 400MB+ of > code to do so. That's good to know, but somewhat unrelated to the issue under discussion. While it is clear from your message that you can build python, as well as python extensions with MingW, it is not that clear whether extensions build with mingw will work in the standard 2.4 distribution, or whether you can use the standard 2.4 distribution to build extensions with mingw. Regards, Martin From pje at telecommunity.com Sun Dec 12 23:21:21 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun Dec 12 23:19:37 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <41BCC116.8040202@v.loewis.de> References: <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl> <BAY23-F336B3D172EA9A1DE45777CABAA0@phx.gbl> Message-ID: <5.1.1.6.0.20041212171632.03786cb0@mail.telecommunity.com> At 11:07 PM 12/12/04 +0100, Martin v. L?wis wrote: >it is not that >clear whether extensions build with mingw will work in the standard >2.4 distribution, or whether you can use the standard 2.4 distribution >to build extensions with mingw. I thought that it was clarified some time ago, with my success reports for the pre-alpha version. Paul Moore has also independently confirmed this, and posted reasons why he doesn't believe that the dangling references to msvcrt should cause a problem in practice. The item that was unclear was whether newer versions of MinGW could get away *without* building a libpython24.a file, and Paul indicated that this was not possible. So, using the script that I posted previously to build libpython24.a from python24.dll is still a valid path for using MinGW to build extensions for Python on Windows. From adam.jtm30 at gmail.com Mon Dec 13 00:18:44 2004 From: adam.jtm30 at gmail.com (Adam Bark) Date: Mon Dec 13 00:18:49 2004 Subject: [Python-Dev] Re: The Other Py2.4 issue? In-Reply-To: <20041212180927.E24C61E4014@bag.python.org> References: <20041212180927.E24C61E4014@bag.python.org> Message-ID: <be4fbf9204121215184b0814b7@mail.gmail.com> Now this might sound a bit stupid but I've only been programming in python for about 6 months and before that about the same on VB. Anyway here goes, as python is built in C & C++ surely every piece of python code has a corresponding piece of C/C++ albeit more complex. So would it be possible to somehow make a program to convert the Python to C & C++ which can then be compiled with a C/C++ compiler. Adam On Sun, 12 Dec 2004 19:09:27 +0100 (CET), python-dev-request@python.org <python-dev-request@python.org> wrote: > Send Python-Dev mailing list submissions to > python-dev@python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/python-dev > or, via email, send a message with subject or body 'help' to > python-dev-request@python.org > > You can reach the person managing the list at > python-dev-owner@python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-Dev digest..." > > > Today's Topics: > > 1. Re: Re: Re: 2.4 news reaches interesting places (Carlos Ribeiro) > 2. Re: The other Py2.4 issue (Paul Moore) > 3. Re: Re: Re: 2.4 news reaches interesting places (Carlos Ribeiro) > 4. Re: Supporting Third Party Modules (was The other Py2.4 > issue) (Bob Ippolito) > 5. Re: The other Py2.4 issue (Carlos Ribeiro) > 6. Re: Re: 2.4 news reaches interesting places (Fredrik Lundh) > 7. Re: The other Py2.4 issue (Martin v. L?wis) > 8. Re: Supporting Third Party Modules (was The other Py2.4 > issue) (Martin v. L?wis) > > > > ---------- Forwarded message ---------- > From: Carlos Ribeiro <carribeiro@gmail.com> > To: Erik Heneryd <erik@heneryd.com> > Date: Sun, 12 Dec 2004 15:14:09 -0200 > Subject: Re: [Python-Dev] Re: Re: 2.4 news reaches interesting places > On Sun, 12 Dec 2004 17:10:58 +0100, Erik Heneryd <erik@heneryd.com> wrote: > > Fredrik Lundh wrote: > > >>>fwiw, IDG's Computer Sweden, "sweden's leading IT-newspaper" has a > > >>>surprisingly big Python article in their most recent issue: > > >>> > > >>> PYTHON FEELS WELL > > >>> Better performance biggest news in 2.4 > > >>> > > > > >>>and briefly interviews swedish zope-developer Johan Carlsson and Python- > > >>>Ware co-founder H?kan Karlsson. > > >> > > > > ... > > > > > > > > so I don't think you can blame Johan or H?kan... the writer simply read the > > > python.org material, and picked a couple of things that he found interesting > > > (decorators and generator expressions may be a big thing for an experienced > > > pythoneer, but they are probably a bit too obscure for a general audience...) > > > > I'm a bit puzzled by the last paragraph, where Python is grouped > > together with PHP and Perl - names starting with p, being popular on > > Linux and not having big, commercial backers. The article then > > concludes "Since Python is copyrighted, it's not truly open. However, > > it can be freely used and redistributed, even commercially." > > > > Huh? Where did THAT come from? You might argue the merits of Python > > being associated with Perl/PHP, but it's a fact that it is. But when it > > is, it's seen as less free? > > The author was probably referring to the old (and as AFAIK already > solved) CRNI copyright issue that ocurred into the 1.x to 2.x series > transition. It's amazing how old memes from Python keep being > remembered and repeated, even years after the fact. It also > illustrates something very important - the community is not doing a > good job at spreading the news; perhaps we talk too much between > ourselves, and too little with the outside market. IMHO the website is > a great part of this, its message being more important to "sell" > Python than the standard library or native .exes. > > About the website, a note from my own experience: when I search for > documentation on Python, I'm usually directed to some of the mirror of > the main python.org site. To find it inside the main site, I have to > use "site:python.org", or even "site:docs.python.org". Usually Google > does a good job at ranking pages, and if it doesn't rank the main > Python website very highly, it's because they're not being referred > to. A campaign to ask people to put links back to the canonical > documentation at the Python website would be nice. > > -- > Carlos Ribeiro > Consultoria em Projetos > blog: http://rascunhosrotos.blogspot.com > blog: http://pythonnotes.blogspot.com > mail: carribeiro@gmail.com > mail: carribeiro@yahoo.com > > > > ---------- Forwarded message ---------- > From: Paul Moore <p.f.moore@gmail.com> > To: Christian Tismer <tismer@stackless.com> > Date: Sun, 12 Dec 2004 17:26:49 +0000 > Subject: Re: [Python-Dev] The other Py2.4 issue > On Sat, 11 Dec 2004 19:57:55 +0100, Christian Tismer > <tismer@stackless.com> wrote: > > Armin Rigo wrote: > > > > Hum, this is getting into a Linux-vs-Windows argument. I don't want to invest > > > time and money on Windows tools just to compile my extension module for > > > Windows users... > > First of all, I'm assuming this is a timing issue. If I understood > your initial posting, your concern was that people wanted Windows > build of your extension *now*, and it was going to take you time to > make it available. > > That's a different issue from you not having the facilities to build > the Windows installers at all, where you rely on 3rd parties making > builds available. > > As Martin points out, ultimately the provision of Windows binaries is > an issue for the extension project - is the demand high enough to > justify the effort, can you find tools and/or a helper, etc etc. > > But the former issue (how quickly you can provide binaries, assuming > that you will do so ultimately) is more relevant for python-dev. > Specifically, because lack of binary extensions can be a barrier to > take-up of the new version. Certainly, in the past, you could pretty > much guarantee that there would be very few Windows users testing beta > releases, because pywin32 binaries weren't available. With 2.4, I have > at least one system I'd upgrade *now* but for the lack of a critical > extension in binary form (I haven't yet had the time to try to adapt > the build process to mingw for myself). > > > Maybe we can set this up as a service? > > That sounds like a good idea. What I'd suggest is needed is a website > where the following can take place: > > 1. People have a way of posting rquests for particular modules. > 2. Installers can be uploaded to satisfy those requests. > 3. There is somewhere to put step-by-step "build it yourself" > instructions, using free components, so that people *without* access > to VS.NET can make progress themselves. Obviously, if a particular > extension can't be built with free compilers, then binaries or access > to VS.NET are the only options. > > The installers should be clearly noted as having no warranty or > support implied, to encourage people to offer binaries they have built > without feeling that they are taking on a support burden. Conversely, > as soon as "official" binaries are available from the extension > project, the binaries available here should be removed (and replaced > with a link to the official site) to reinforce the "unofficial" nature > of the binaries provided here. > > The biggest potential issue with such a site is clearly validation. > I've no idea how to make something like this work without it being a > major virus risk. Which may, sadly, be enough to kill the idea :-( > > Paul. > > > > ---------- Forwarded message ---------- > From: Carlos Ribeiro <carribeiro@gmail.com> > To: "python-dev@python.org" <python-dev@python.org> > Date: Sun, 12 Dec 2004 15:32:03 -0200 > Subject: Re: [Python-Dev] Re: Re: 2.4 news reaches interesting places > Hello all, > > Just to complement my previous remarks, I would like to point out how > do a competing language defines itself in its own website. The > perl.org website has a simple faq that is a good piece of marketing. > What follows are direct quotes, just to point out how ot handle the > market perception about their quality & speed. > > -- "Perl takes the best features from other languages, such as C, awk, > sed, sh, and BASIC, among others." > > (A claim can't possibly be any more generic than this. Strangely > enough, it mentions only older languages -- not Java, C++, or even > Python (!). This is possibly a sign of an old quote, but anyway: it > has a good marketing effect, specially for non-techies.) > > -- "Perl can be embedded into web servers to speed up processing by as > much as 2000%." > > (BTW, this quote is embarrassing misleading -- it probably means that > Perl is 20x slower when started on a request basis by the web server, > and that embedding it will accelerate response by a huge factor. I'm > sure non-techies will read it as "Perl is able to accelerate my server > 20x!") > > Of course, the point here is not Perl-bashing. The point here is that > we should be able to "sell" Python better than we do now, even without > the need to resort to such poor measures. I'm sure the Python > community does have good & creative people that can write a good > "selling" FAQ for Python, emphasizing the main points of the language. > > For those who believe that a non-profit project should not do any > marketing, a reminder. If the perception about Python is one of a slow > language, it's much more difficult to find places where you can use > Python. In the long run, many of us may be forced to work with other > languages & tools, just because that's where the money is. I > personally take it a matter of personal interest, because I know how > hard it is to "sell" Python to companies here in Brazil. > > -- > Carlos Ribeiro > Consultoria em Projetos > blog: http://rascunhosrotos.blogspot.com > blog: http://pythonnotes.blogspot.com > mail: carribeiro@gmail.com > mail: carribeiro@yahoo.com > > > > ---------- Forwarded message ---------- > From: Bob Ippolito <bob@redivi.com> > To: "Martin v. L?wis" <martin@v.loewis.de> > Date: Sun, 12 Dec 2004 12:46:18 -0500 > Subject: Re: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) > > On Dec 12, 2004, at 11:02 AM, Martin v. L?wis wrote: > > > Bob Ippolito wrote: > >>> I believe this is not implementable: How can the DLL know which > >>> Python > >>> DLL to use? > >> Well for py2app on Mac OS X, I wrote an executable stub that chooses > >> a Python runtime from an XML file, looks up and binds a few symbols > >> from it dynamically, and then starts doing stuff. > > > > While that would work, I think this is inappropriate for this specific > > issue: we want to write extension modules which are independent of > > the Python version, and might even be used with multiple Python > > installations on the same system. In that case, adding configuration > > files won't work, as each usage of the extension might require a > > different Python DLL. > > Yes, of course, I was talking about the executable, not extensions. On > Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows > extensions to link to no Python whatsoever. The extensions will just > find the symbols it needs from some other image already loaded into the > process at runtime. If it weren't for the "forced" ABI > incompatibility, we'd already have extensions that work > cross-Python-major-version (assuming they used a safe subset of > functions and structures). > > -bob > > > > ---------- Forwarded message ---------- > From: Carlos Ribeiro <carribeiro@gmail.com> > To: Martin v. L?wis <martin@v.loewis.de> > Date: Sun, 12 Dec 2004 15:53:10 -0200 > Subject: Re: [Python-Dev] The other Py2.4 issue > On Sun, 12 Dec 2004 16:40:22 +0100, Martin v. L?wis <martin@v.loewis.de> wrote: > > If none of your users volunteers to do the build for you, I would stop > > worrying about the Windows users. > > Sorry, Martin. I understand your point, but I think you are not being > realistic. I for myself took the decision to use only free tools for > my own development, but I still have to suport my Windows customers. I > can't force them to change to Linux. I don't own a copy of MSVC. Also, > one of the reasons to choose a third part module is to save time. The > moment I am required to recompile everything myself I'm losing this > convenience. This of course impacts my ability to focus on my own > work, and so the story goes. > > I'm not saying that module authors should work for free just to save > me some time & hassle. It's fair if an author decides to release a > Linux-only module. But again -- this is not realistic. The world has a > lot of Windows users, and I depend on them for my own income. If I > can't find a good set of Python tools for my projects, what should I > do? Picking another language is not a choice, mind you :-) > > All in all, I sincerely hope that this discussion end up in a high > note. I'm not qualified to talk about the particulars of C compilers & > development environments, but as a Python user, I have hope that a > good solution will be found to make the process of building Python > extensions for Windows more convenient. The dream scenario is not to > require recompiling, at least inside the same major release (2.4 to > 2.5, for example). That would be really great. > > -- > Carlos Ribeiro > Consultoria em Projetos > blog: http://rascunhosrotos.blogspot.com > blog: http://pythonnotes.blogspot.com > mail: carribeiro@gmail.com > mail: carribeiro@yahoo.com > > > > ---------- Forwarded message ---------- > From: "Fredrik Lundh" <fredrik@pythonware.com> > To: python-dev@python.org > Date: Sun, 12 Dec 2004 18:53:27 +0100 > Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places > > . Multiple assignment is slower than individual assignment. For > > example "x,y=a,b" is slower than "x=a; y=b". However, multiple > > assignment is faster for variable swaps. For example, "x,y=y,x" is > > faster than "t=x; x=y; y=t". > > marginally faster in 2.4, a lot slower in earlier versions. maybe you > should mark sections that rely on 2.4-specific optimizations, so that > people who use earlier versions don't end up slowing their programs > down... > > </F> > > > > ---------- Forwarded message ---------- > From: "Martin v. L?wis" <martin@v.loewis.de> > To: Carlos Ribeiro <carribeiro@gmail.com> > Date: Sun, 12 Dec 2004 19:06:44 +0100 > Subject: Re: [Python-Dev] The other Py2.4 issue > Carlos Ribeiro wrote: > >>If none of your users volunteers to do the build for you, I would stop > >>worrying about the Windows users. > > > > > > Sorry, Martin. I understand your point, but I think you are not being > > realistic. I for myself took the decision to use only free tools for > > my own development, but I still have to suport my Windows customers. I > > can't force them to change to Linux. I don't own a copy of MSVC. Also, > > one of the reasons to choose a third part module is to save time. The > > moment I am required to recompile everything myself I'm losing this > > convenience. This of course impacts my ability to focus on my own > > work, and so the story goes. > > I did not suggest that *all* your Windows users should recompile > your module - just a single one would be sufficient. > > > I'm not saying that module authors should work for free just to save > > me some time & hassle. It's fair if an author decides to release a > > Linux-only module. But again -- this is not realistic. The world has a > > lot of Windows users, and I depend on them for my own income. If I > > can't find a good set of Python tools for my projects, what should I > > do? Picking another language is not a choice, mind you :-) > > As I said: Find a volunteer that has the necessary build infrastructure, > and have that volunteer build the extension for you. > > > The dream scenario is not to > > require recompiling, at least inside the same major release (2.4 to > > 2.5, for example). That would be really great. > > That is guaranteed. Extensions built for 2.4 will certainly continue > to work in 2.4.1, and later 2.4.x. They will stop working with 2.5 > (as they are linked with python24.dll). > > Regards, > Martin > > > > ---------- Forwarded message ---------- > From: "Martin v. L?wis" <martin@v.loewis.de> > To: Bob Ippolito <bob@redivi.com> > Date: Sun, 12 Dec 2004 19:09:05 +0100 > Subject: Re: [Python-Dev] Supporting Third Party Modules (was The other Py2.4 issue) > Bob Ippolito wrote: > > Yes, of course, I was talking about the executable, not extensions. On > > Mac OS X 10.3+, the linker flag -undefined dynamic_lookup allows > > extensions to link to no Python whatsoever. > > It's the same on SysV ELF shared libraries, and in most other unices. > > > The extensions will just > > find the symbols it needs from some other image already loaded into the > > process at runtime. If it weren't for the "forced" ABI incompatibility, > > we'd already have extensions that work cross-Python-major-version > > (assuming they used a safe subset of functions and structures). > > Are you talking about a forced ABI incompatibility, beyond the Windows > issue of linking with a specific pythonxy.dll? > > On Unix, you certainly can have extensions across Python major versions. > > Regards, > Martin > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > > > From abkhd at hotmail.com Mon Dec 13 00:19:55 2004 From: abkhd at hotmail.com (A.B., Khalid) Date: Mon Dec 13 00:22:58 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue Message-ID: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> Martin v. Löwis wrote: >That's good to know, but somewhat unrelated to the issue under >discussion. While it is clear from your message that you can build >python, as well as python extensions with MingW, it is not that >clear whether extensions build with mingw will work in the standard >2.4 distribution, or whether you can use the standard 2.4 distribution >to build extensions with mingw. Martin it is somewhat related. :) But seriously, I did mention that which you note, and this is partly the reason why I am following this thread. But you are right. We must resolve that issue as it pertains to the pyMinGW patched & MinGW compiled Python, if you would kindly allow me to. And so I propose the following: [1] Can someone please email me [*] the officially distributed Python24.dll and python.exe, (I do not wish to install the full version yet; if this is too much to ask, then nevermind, I'll download the whole distribution), or alternatively, and more conveniently [2] Can someone who has the official Python 2.4 download the sample extension [**] created using the pyMinGW patched & MinGW compiled Python 2.4 and SWIG? And see if it works? Sources are in the zip file whose details appears bellow. Regards, Khalid [*] Email to this address please: abkhd[-AT-]earth.co.jp [**] pymingw.zip (5.35 KB): Location: http://jove.prohosting.com/iwave/misc/pymingw.zip MD5: b4c1a2ebcb8a00fde89f6efe102d983f -------------------------- Contents in KB: -------------------------- _PYMINGW PYD 9.216 13/12/04 1:00 _pymingw.pyd PYMINGW PY 1.115 13/12/04 1:00 pymingw.py PYMINGW I 278 13/12/04 1:00 pymingw.i EXAMPLE C 848 13/12/04 0:23 example.c SETUP PY 192 13/12/04 0:59 setup.py _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ From tdelaney at avaya.com Mon Dec 13 01:11:47 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Mon Dec 13 01:11:53 2004 Subject: [Python-Dev] Re: The Other Py2.4 issue? Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE72121A@au3010avexu1.global.avaya.com> Adam Bark wrote: > Now this might sound a bit stupid but I've only been programming in > python for about 6 months and before that about the same on VB. Anyway > here goes, as python is built in C & C++ surely every piece of python > code has a corresponding piece of C/C++ albeit more complex. So would > it be possible to somehow make a program to convert the Python to C & > C++ which can then be compiled with a C/C++ compiler. Yes. And it would run at about the same speed as CPython. The only way to significantly improve this is to remove much of the dynamism of Python. As an example, I've been playing around with Raymond's constant binding - making it do more aggressive binding (I'll give an example later). By binding constants, I'm able to reduce the runtimes for psyco-compiled versions of friendly code (lots of things that can be converted into constant references) from around 2 seconds to less than 0.001 seconds. That's a very significant speedup. Unfortunately, it doesn't translate as well into real-world code - or even benchmarks (parrotbench for example gets a slight speedup but it's not overly significant). As a quick example if the changes I'm playing with, def func() docutils.parsers.rst.states.struct.__init__ Raymond's generates: JUMP_FORWARD 0 LOAD_CONST (docutils) LOAD_ATTR (parsers) LOAD_ATTR (rst) LOAD_ATTR (states) LOAD_ATTR (struct) LOAD_ATTR (__init__) Mine generates 0 JUMP_ABSOLUTE 15 3 NOP 4 NOP 5 NOP 6 NOP 7 NOP 8 NOP 9 NOP 10 NOP 11 NOP 12 NOP 13 NOP 14 NOP 15 LOAD_CONST (<unbound method Struct.__init__) I've also been doing some peephole optimisations - the above actually ends up being an immediate jump to the return bytecode, but that's fairly useless. Using JUMP_ABSOLUTE instead of JUMP_FORWARD simplifies doing this. Once I've got the code into a fit state I'll send it to you if you're interested Raymond ... Tim Delaney From barry at python.org Mon Dec 13 02:36:45 2004 From: barry at python.org (Barry Warsaw) Date: Mon Dec 13 02:36:48 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <864d3709041212093274becde2@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> Message-ID: <1102901805.9393.104.camel@geddy.wooz.org> On Sun, 2004-12-12 at 12:32, Carlos Ribeiro wrote: > For those who believe that a non-profit project should not do any > marketing, a reminder. If the perception about Python is one of a slow > language, it's much more difficult to find places where you can use > Python. In the long run, many of us may be forced to work with other > languages & tools, just because that's where the money is. I > personally take it a matter of personal interest, because I know how > hard it is to "sell" Python to companies here in Brazil. Actually, there's another problem in the corporate world that has nothing to do with Python's performance (at least not directly). When a manager has to hire 25 programmers for a project they think to themselves, "well, Java programmers are a dime a dozen so I'll have no problem finding warm bodies if we write it in Java. Can I even /find/ 25 Python programmers?" -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041212/5b10e7ae/attachment.pgp From carribeiro at gmail.com Mon Dec 13 03:13:15 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Mon Dec 13 03:13:18 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <1102901805.9393.104.camel@geddy.wooz.org> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <1102901805.9393.104.camel@geddy.wooz.org> Message-ID: <864d37090412121813379f6aeb@mail.gmail.com> On Sun, 12 Dec 2004 20:36:45 -0500, Barry Warsaw <barry@python.org> wrote: > On Sun, 2004-12-12 at 12:32, Carlos Ribeiro wrote: > > > For those who believe that a non-profit project should not do any > > marketing, a reminder. If the perception about Python is one of a slow > > language, it's much more difficult to find places where you can use > > Python. In the long run, many of us may be forced to work with other > > languages & tools, just because that's where the money is. I > > personally take it a matter of personal interest, because I know how > > hard it is to "sell" Python to companies here in Brazil. > > Actually, there's another problem in the corporate world that has > nothing to do with Python's performance (at least not directly). When a > manager has to hire 25 programmers for a project they think to > themselves, "well, Java programmers are a dime a dozen so I'll have no > problem finding warm bodies if we write it in Java. Can I even /find/ > 25 Python programmers?" You're right, specially for big corporations. But in the end, we're just running in circles: it's hard to get new programmers to learn Python, partly because it's in low demand, and partly because the language has an totally undeserved fame of being slow. That's right - when I talk to fellow programmers that I'm writing software in Python, many of them are amazed and ask me, "but isn't it slow?". I've heard it more than once... having some place that I could point them out so they could check it for themselves (perhaps "www.pythonspeed.com"?) would be *great*. IMHO, Python performance is not an issue 99% of the time. I swear I can say the same about C++. The difference between C++ and Python is not how fast they are relatively to each other, but *where* the slow part is. Anyone who tried to write complex code in any other language knows that most programmers usually resort to slow but easy to implement algorithms for things such as sorting, list handling, etc; these parts of the code are naturally fast in Python, while others may be faster in C++... so we're just trading "6 for half a dozen", as we say in portuguese :-) -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From p.f.moore at gmail.com Mon Dec 13 10:58:49 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Mon Dec 13 10:58:52 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> Message-ID: <79990c6b041213015861230c01@mail.gmail.com> On Sun, 12 Dec 2004 23:19:55 +0000, A.B., Khalid <abkhd@hotmail.com> wrote: > [2] Can someone who has the official Python 2.4 download the sample > extension [**] created using the pyMinGW patched & MinGW compiled Python 2.4 > and SWIG? And see if it works? > Sources are in the zip file whose details appears bellow. I tried this out, and from some basic uses, it seems to work OK. However, the PYD file references msvcrt.dll, which suggests that there will be issues in more complex cases. The biggest problem with CRT compatibility issues is that (AFAIK) no-one has actually been able to trigger a *real* error, all of the problems are still theoretical. I have done some fairly extensive analysis of what could go wrong, and I suspect that there are very few genuine cases where data from one CRT could end up being manipulated by another - but this doesn't mean that the problems aren't real, just VERY hard to locate :-( My current technique for checking an extension is compatible with Python 2.4 is to run objdump -p (from the mingw distribution - use dumpbin /imports from MSVC) and review the import table. If any symbols are referenced from msvcrt.dll, you need to convince yourself that they are used solely by the mingw runtime startup code. For added certainty, add a dummy function to your C code which references these symbols, rebuild and confirm that they are now satisfied from msvcr71. (For total guarantees, you need to get Martin to validate your analysis, as he seems to be the only person here who *really* understands the potential issues :-)) If you want to build extensions compatible with the standard Python 2.4 build, you need to add the -lmsvcr71 flag to the build (link step). Hope this helps, Paul. From theller at python.net Mon Dec 13 12:36:23 2004 From: theller at python.net (Thomas Heller) Date: Mon Dec 13 12:35:18 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <79990c6b041213015861230c01@mail.gmail.com> (Paul Moore's message of "Mon, 13 Dec 2004 09:58:49 +0000") References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> Message-ID: <wtvma0qw.fsf@python.net> Paul Moore <p.f.moore@gmail.com> writes: > The biggest problem with CRT compatibility issues is that (AFAIK) > no-one has actually been able to trigger a *real* error, all of the > problems are still theoretical. There have been problems with the bdist_wininst exe-stub linking to the wrong CRT dll, but I don't remember the exact details. And recently I played with bindings to OpenGL's glut32.dll - glut calls exit() from internal C code. If linked with the wrong CRT, this will do nothing instead of terminating the process. Thomas From mal at egenix.com Mon Dec 13 14:48:36 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Dec 13 14:48:39 2004 Subject: [Python-Dev] Deprecated xmllib module In-Reply-To: <41B2C870.5090609@v.loewis.de> References: <001801c4da72$543801a0$ab2fc797@oemcomputer> <41B2C870.5090609@v.loewis.de> Message-ID: <41BD9DB4.6040501@egenix.com> Martin v. L?wis wrote: > As for PEP 4: I don't know whether it needs to be listed there. It > appears that the PEP is largely unmaintained (I, personally, do not > really maintain it). So one option would be to just stop using PEP 4 > for recording deprecations, since we now have the warnings module. > If we want to keep PEP 4, we need to follow the procedures it > requires (or modify them if we don't like them). There is a need for a central place where deprecations are posted. You can't tell people to go hunt the standard lib for uses of the warning module in order for them to catch up with the current state of affairs. One thing that we shouldn't forget is that Python users do not necessarily upgrade to new Python releases every time there's a new release. Production users of Python will often skip a release or two depending on their needs, time frames and audit cycles. It is very important for them to know that module or other technology will be deprecated two releases down the road, so that they can accomodate for these changes today. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 13 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2004-12-06: Released eGenix mx Extensions for Python 2.4 ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Mon Dec 13 15:13:13 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Dec 13 15:13:15 2004 Subject: [Python-Dev] Rewriting PEP4 In-Reply-To: <1102370092.9565.30.camel@geddy.wooz.org> References: <000701c4dab6$cf5e8e00$3b01a044@oemcomputer> <200412060243.01129.anthony@interlink.com.au> <41B34B62.7040201@v.loewis.de> <41B39EA4.9030105@ocf.berkeley.edu> <41B4CF07.4050108@v.loewis.de> <1102370092.9565.30.camel@geddy.wooz.org> Message-ID: <41BDA379.7030403@egenix.com> Barry Warsaw wrote: > On Mon, 2004-12-06 at 16:28, "Martin v. L?wis" wrote: > > Martin, +1 on everything you wrote, with one minor quibble. > > >>Removal >>======= >>If the module has been deprecated for atleast a year and atleast >>a version, it can be removed. Removal should move it to old-libs >>for pure Python modules; a removal procedure for pure C modules >>has not been defined yet. > > > I wonder if the one year/one version rule is too short. Given that new > versions come about every 18 months, I'd probably give it a 2 year/one > version limit. +1 I'd make that 2 years / 2 versions, though. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 13 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2004-12-06: Released eGenix mx Extensions for Python 2.4 ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From sdeibel at wingware.com Mon Dec 13 17:30:45 2004 From: sdeibel at wingware.com (Stephan Deibel) Date: Mon Dec 13 17:29:37 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <864d37090412121813379f6aeb@mail.gmail.com> Message-ID: <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog> On Mon, 13 Dec 2004, Carlos Ribeiro wrote: > On Sun, 12 Dec 2004 20:36:45 -0500, Barry Warsaw <barry@python.org> wrote: > > Actually, there's another problem in the corporate world that has > > nothing to do with Python's performance (at least not directly). When a > > manager has to hire 25 programmers for a project they think to > > themselves, "well, Java programmers are a dime a dozen so I'll have no > > problem finding warm bodies if we write it in Java. Can I even /find/ > > 25 Python programmers?" > > You're right, specially for big corporations. But in the end, we're > just running in circles: it's hard to get new programmers to learn > Python, partly because it's in low demand, and partly because the > language has an totally undeserved fame of being slow. The perception-of-speed issue is clearly important but we're definately not running in circles. There are quite a few signs that the Python user base is expanding substantially. For example, a September article in InfoWorld said "But the big winner this time around is the object-oriented scripting language Python, which saw a 6 percent gain in popularity, almost doubling last year's results." http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature Also, there are companies that have hundreds of Python programmers, like some of those that have done success stories: http://www.pythonology.org/success That doesn't mean the perception that you can't hire 25 at once isn't a problem, but clearly some companies know that turning someone into a Python programmer is easy enough to offset the smaller available pool. To counter speed claims, look at articles like this one: http://www.pythonology.org/success&story=suzanne "Python helps AFNIC manage over 10,000 internet domain name registration requests per minute in a landrush for the ".fr" top-level internet domain" Yes it would be nice to have more of these, where performance is mentioned in the summary! Please contact me if you can contribute one. BTW, I can't resist my own favorite speed anecdote: I once wrote a one-off script to process down a couple of gigabytes of variously fragmented web logs into month-by-month files. I thought I was being naive doing f.readline() in a for loop with some date parsing code for each entry. But I was completely astounded how fast it processed -- and it just worked the first time around. - Stephan From abkhd at hotmail.com Mon Dec 13 17:43:18 2004 From: abkhd at hotmail.com (A.B., Khalid) Date: Mon Dec 13 17:44:18 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue Message-ID: <BAY23-F8386F00CD4776C6ACCB87ABAB0@phx.gbl> Paul Moore wrote: >I tried this out, and from some basic uses, it seems to work OK. >However, the PYD file references msvcrt.dll, which suggests that there >will be issues in more complex cases. The biggest problem with CRT >compatibility issues is that (AFAIK) no-one has actually been able to >trigger a *real* error, all of the problems are still theoretical. I >have done some fairly extensive analysis of what could go wrong, and I >suspect that there are very few genuine cases where data from one CRT >could end up being manipulated by another - but this doesn't mean that >the problems aren't real, just VERY hard to locate :-( First of all, thank you for your input. It is greatly appreciated. And yes if Martin, for whom I have much respect, can help here, then that would be awesome. Okay. This is partly my own fault. I am not using a standard MinGW distribution. This is why there is no release date in the sys.version information of the Python I compiled. I am using GCC RC 3.4.1 but the linking is still done by the 2002 MinGW 3.2, which is most probably hard- wired to use msvcrt.dll. Having said that, however, let me quickly add that it is almost a certainity that the newer standard MinGW distributions link with the latest libraries. Nevertheless, I updated pyMinGW's cygwinccompiler.py to always link in with libmsvcr71, and rebuit the sample pymingw extension[1]. Now according to objdump we get the following: [CUT] 00007014 000070b0 00000000 00000000 00007420 00007158 DLL Name: KERNEL32.dll vma: Hint/Ord Member-Name Bound-To 72fc 0 AddAtomA 7308 147 FindAtomA 7314 191 GetAtomNameA 00007028 000070c4 00000000 00000000 00007468 0000716c DLL Name: msvcr71.dll vma: Hint/Ord Member-Name Bound-To 7324 56 __dllonexit 7334 189 _errno 7340 610 ctime 7348 619 fflush 7354 630 fputs 735c 634 free 7364 642 fwrite 7370 685 malloc 737c 701 puts 7384 724 strcmp 7390 726 strcpy 739c 730 strlen 73a8 733 strncpy 73b4 748 time 0000703c 00007104 00000000 00000000 00007478 000071ac DLL Name: msvcrt.dll vma: Hint/Ord Member-Name Bound-To 73bc 510 abort 00007050 00000000 00000000 00000000 00000000 00000000 There is an export table in .edata at 0x6000 [CUT] So what see ye? :) Does it look good? Best wishes Khalid [1] pymingw-01.zip (5.37 KB): Location: http://jove.prohosting.com/iwave/misc/pymingw-01.zip MD5: 63b486093b344b0299538c952dea4ce1 -------------------------- Contents in KB: -------------------------- PYMINGW PY 1.115 13/12/04 16:12 pymingw.py PYMINGW I 278 13/12/04 1:00 pymingw.i EXAMPLE C 848 13/12/04 0:23 example.c SETUP PY 192 13/12/04 0:59 setup.py _PYMINGW PYD 9.216 13/12/04 16:12 _pymingw.pyd _________________________________________________________________ Don't just search. Find. Check out the new MSN Search! http://search.msn.click-url.com/go/onm00200636ave/direct/01/ From jhylton at gmail.com Mon Dec 13 17:49:03 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Mon Dec 13 17:49:08 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog> References: <864d37090412121813379f6aeb@mail.gmail.com> <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog> Message-ID: <e8bf7a53041213084958aba266@mail.gmail.com> On Mon, 13 Dec 2004 11:30:45 -0500 (EST), Stephan Deibel <sdeibel@wingware.com> wrote: > For example, a September article in InfoWorld said "But the big winner > this time around is the object-oriented scripting language Python, which > saw a 6 percent gain in popularity, almost doubling last year's results." > > http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature Can we extrapolate from the numbers here to get an estimate of how many Python developers there are? I was asked for that number at workshop a few months ago and I didn't have any idea how to answer. Is there a good answer? Two possibilities come to mind. 1) 14% of developers in the survey work at companies that use Python. How many developers are there? Assume that 14% of them use Python. But what's a good estimate for "number of developers." Pretty rough -- number of survey respondents at company != number of Python programmers at company, and %age companies != %age of programmers. 2) 64% of companies use Java, 4.5 times more than Python. Find out how many Java programmers there are, divide by 4.5. Jeremy From p.f.moore at gmail.com Mon Dec 13 17:52:45 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Mon Dec 13 17:52:48 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <BAY23-F8386F00CD4776C6ACCB87ABAB0@phx.gbl> References: <BAY23-F8386F00CD4776C6ACCB87ABAB0@phx.gbl> Message-ID: <79990c6b0412130852458e5f0b@mail.gmail.com> On Mon, 13 Dec 2004 16:43:18 +0000, A.B., Khalid <abkhd@hotmail.com> wrote: > So what see ye? :) Does it look good? Looks good to me. The one remaining link to msvcrt, abort, is likely to be due to the startup code using it, and your code not referencing that symbol, so that it doesn't force linking from msvcr71. To check this, you could add a dummy call to abort() somewhere in your code and this last reference should vanish. So this now looks just like an extension built using --compiler=mingw32 with the standard Python 2.4 build, which I guess means that your build can create extensions that are interoperable with both itself and the standard Python 2.4. Paul. From fredrik at pythonware.com Mon Dec 13 18:07:04 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Dec 13 18:07:16 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places References: <864d37090412121813379f6aeb@mail.gmail.com> <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog> Message-ID: <cpki7r$k3p$1@sea.gmane.org> Stephan Deibel wrote: > BTW, I can't resist my own favorite speed anecdote: I once wrote a > one-off script to process down a couple of gigabytes of variously > fragmented web logs into month-by-month files. I thought I was being > naive doing f.readline() in a for loop with some date parsing code for > each entry. But I was completely astounded how fast it processed -- and > it just worked the first time around. I was doing the same thing today (well, processing system logs, not web logs) and was completely baffled when I found that a program that just worked the first time around on our development machine, crashed with a MemoryError on the production machine. oh well. </F> From FBatista at uniFON.com.ar Mon Dec 13 18:09:03 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Mon Dec 13 18:12:00 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places Message-ID: <A128D751272CD411BC9200508BC2194D053C7CDA@escpl.tcp.com.ar> [Stephan Deibel] #- For example, a September article in InfoWorld said "But the #- big winner #- this time around is the object-oriented scripting language #- Python, which #- saw a 6 percent gain in popularity, almost doubling last #- year's results." How big are the chances that SourceForge numbers actually could be extrapolated to the rest of the universe? According to them (check software map, and look by programming language), and showing everything with a "developers choice for their project" share >= 2%: C++ 18.5 C 18.1 Java 17.5 PHP 12.9 Perl 7.2 Python 4.7 Visual Basic 2.6 C# 2.6 JavaScript 2.6 Delphi/Kylix 2.1 Unix Shell 2.0 It also would nice to see a graph showing tendencies here. . Facundo -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20041213/08fa6e09/attachment.htm From sdeibel at wingware.com Mon Dec 13 19:11:38 2004 From: sdeibel at wingware.com (Stephan Deibel) Date: Mon Dec 13 19:10:32 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <e8bf7a53041213084958aba266@mail.gmail.com> References: <864d37090412121813379f6aeb@mail.gmail.com> <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog> <e8bf7a53041213084958aba266@mail.gmail.com> Message-ID: <Pine.LNX.4.50.0412131300240.3163-100000@hedgehog> On Mon, 13 Dec 2004, Jeremy Hylton wrote: > > http://www.infoworld.com/article/04/09/24/39FErrdev_1.html?s=feature > > Can we extrapolate from the numbers here to get an estimate of how > many Python developers there are? I was asked for that number at > workshop a few months ago and I didn't have any idea how to answer. > Is there a good answer? > > Two possibilities come to mind. 1) 14% of developers in the survey > work at companies that use Python. How many developers are there? > Assume that 14% of them use Python. But what's a good estimate for > "number of developers." Pretty rough -- number of survey respondents > at company != number of Python programmers at company, and %age > companies != %age of programmers. Supposedly there are 5-6 million developers world-wide (by various estimates; I've no idea whether to believe them). If you just multiply out naively you get 700K-840K Python programmers. There certainly are vastly more one person Python projects than large ones, so this may not be all that far off. > 2) 64% of companies use Java, 4.5 > times more than Python. Find out how many Java programmers there are, > divide by 4.5. I've seen claims of 3-4 million java programmers so that's 666K-888K Python programmers. Interestingly, this matches up with the above abuse of statistics. Independently by various other techniques of wildly guesstimating over the years, I've come to a current figure of about 350K serious Python projects world-wide. Many are single-person projects so this does mesh w/in an order of magnitude with the above. - Stephan From sdeibel at wingware.com Mon Dec 13 19:24:58 2004 From: sdeibel at wingware.com (Stephan Deibel) Date: Mon Dec 13 19:23:50 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <A128D751272CD411BC9200508BC2194D053C7CDA@escpl.tcp.com.ar> Message-ID: <Pine.LNX.4.50.0412131315580.3163-100000@hedgehog> On Mon, 13 Dec 2004, Batista, Facundo wrote: > [Stephan Deibel] > > #- For example, a September article in InfoWorld said "But the > #- big winner > #- this time around is the object-oriented scripting language > #- Python, which > #- saw a 6 percent gain in popularity, almost doubling last > #- year's results." > > How big are the chances that SourceForge numbers actually could be > extrapolated to the rest of the universe? Not very good, I think. I suspect the vast majority of programmers have never heard of source forge. I'm actually surprised that it's only 4.7% on source forge -- that seems to indicate Python is doing quite a bit better in the non-open source world than on SF. Interesting... wouldn't be surprised if this is because the speed myth has stronger hold among hacker types than business programmer types. If people feel this is getting off-topic for python-dev, there is also the mostly dormant marketing-python list: http://wingware.com/mailman/listinfo/marketing-python (the trolls have been removed) - Stephan From nidoizo at yahoo.com Mon Dec 13 20:13:06 2004 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Mon Dec 13 20:13:19 2004 Subject: [Python-Dev] Re: 2.4 news reaches interesting places In-Reply-To: <001d01c4df09$87e5caa0$e841fea9@oemcomputer> References: <e8bf7a530412101128326aea7b@mail.gmail.com> <001d01c4df09$87e5caa0$e841fea9@oemcomputer> Message-ID: <cpkpk0$e0v$1@sea.gmane.org> Raymond Hettinger wrote: > I felt the same way when reading it. Also, it seemed to embody the > political outlook that optimization is sinful. The document could be > much more positive, fact based, and informative. Also, the wording > seems somewhat outdated. > > A draft for a new entry is included below. Review and feedback are > requested. I would add a reference to __slots__, particularly for wrappers of objects. Regards, Nicolas From jhylton at gmail.com Mon Dec 13 21:08:56 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Mon Dec 13 21:09:00 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <Pine.LNX.4.50.0412131300240.3163-100000@hedgehog> References: <864d37090412121813379f6aeb@mail.gmail.com> <Pine.LNX.4.50.0412131050190.3163-100000@hedgehog> <e8bf7a53041213084958aba266@mail.gmail.com> <Pine.LNX.4.50.0412131300240.3163-100000@hedgehog> Message-ID: <e8bf7a5304121312086d13839c@mail.gmail.com> On Mon, 13 Dec 2004 13:11:38 -0500 (EST), Stephan Deibel <sdeibel@wingware.com> wrote: > On Mon, 13 Dec 2004, Jeremy Hylton wrote: > > Two possibilities come to mind. 1) 14% of developers in the survey > > work at companies that use Python. How many developers are there? > > Assume that 14% of them use Python. But what's a good estimate for > > "number of developers." Pretty rough -- number of survey respondents > > at company != number of Python programmers at company, and %age > > companies != %age of programmers. > > Supposedly there are 5-6 million developers world-wide (by various > estimates; I've no idea whether to believe them). If you just multiply > out naively you get 700K-840K Python programmers. There certainly are > vastly more one person Python projects than large ones, so this may not be > all that far off. > > > 2) 64% of companies use Java, 4.5 > > times more than Python. Find out how many Java programmers there are, > > divide by 4.5. > > I've seen claims of 3-4 million java programmers so that's 666K-888K > Python programmers. Interestingly, this matches up with the above abuse > of statistics. Thanks! I didn't know the numbers from which to guestimate, and it's nice that they are in agreement. This seems like a satisfactory back-of-the-envelope answer. Jeremy From greg at electricrain.com Mon Dec 13 21:41:38 2004 From: greg at electricrain.com (Gregory P. Smith) Date: Mon Dec 13 21:41:42 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <20041213204138.GH23061@zot.electricrain.com> On Wed, Dec 08, 2004 at 02:18:48PM -0800, Guido van Rossum wrote: > I was pleasantly surprised to find a pointer to this article in a news > digest that the ACM emails me regularly (ACM TechNews). > > http://gcn.com/vol1_no1/daily-updates/28026-1.html > > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? All high level dynamic languages need to fight this. Substitute python for perl, php, ruby, tcl and the statements in that article still apply. They'll all have slower execution times (on avg) their compiled or JITed bretheren. I suggest reading that article in a different light: The point of view of the people who are likely to believe what they read in government computer news is that it just confirmed that python is a useful languange for getting the job done easily. Not that one shouldn't use it because "its slow." -- All new Python 2.4, now with blue crystals! From janssen at parc.com Mon Dec 13 21:57:07 2004 From: janssen at parc.com (Bill Janssen) Date: Mon Dec 13 21:57:39 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: Your message of "Sun, 12 Dec 2004 18:13:15 PST." <864d37090412121813379f6aeb@mail.gmail.com> Message-ID: <04Dec13.125717pst."58617"@synergy1.parc.xerox.com> > That's right - > when I talk to fellow programmers that I'm writing software in Python, > many of them are amazed and ask me, "but isn't it slow?". I've heard > it more than once... I heard it last month. In the last couple of months, an acquaintance of mine has been trying out Python. He likes the language, particularly the support for character sets, but has (apparently) given up after benchmarking a typical (for him) task against Perl. The task is to use the "re" package to heavily modify a file by a sequence of re substitutions. Apparently the Python program, which applies the same re substitutions in the same order as the Perl program, takes 3 times as long to run. He thinks it's because of mutable strings in Perl -- that is, he thinks the string being modified (which is long, a whole file full of text) is modified in place in Perl, but has to be re-consed in Python. Bill From imbaczek at gmail.com Mon Dec 13 22:03:36 2004 From: imbaczek at gmail.com (=?UTF-8?Q?Marek_=22Baczek=22_Baczy=C5=84ski?=) Date: Mon Dec 13 22:03:40 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <7143231425898203248@unknownmsgid> References: <864d37090412121813379f6aeb@mail.gmail.com> <7143231425898203248@unknownmsgid> Message-ID: <5f3d2c31041213130317323b15@mail.gmail.com> On Mon, 13 Dec 2004 12:57:07 PST, Bill Janssen <janssen@parc.com> wrote: > Apparently the Python program, which applies the same re substitutions > in the same order as the Perl program, takes 3 times as long to run. > He thinks it's because of mutable strings in Perl -- that is, he > thinks the string being modified (which is long, a whole file full of > text) is modified in place in Perl, but has to be re-consed in Python. Did he use re.compile? -- { Marek Baczy?ski :: UIN 57114871 :: GG 161671 :: JID imbaczek@jabber.gda.pl } { http://www.vlo.ids.gda.pl/ | imbaczek at poczta fm | http://www.promode.org } .. .. .. .. ... ... ...... evolve or face extinction ...... ... ... .. .. .. .. From martin at v.loewis.de Mon Dec 13 23:17:51 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Dec 13 23:17:53 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <79990c6b041213015861230c01@mail.gmail.com> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> Message-ID: <41BE150F.9050100@v.loewis.de> Paul Moore wrote: > My current technique for checking an extension is compatible with > Python 2.4 is to run objdump -p (from the mingw distribution - use > dumpbin /imports from MSVC) and review the import table. If any > symbols are referenced from msvcrt.dll, you need to convince yourself > that they are used solely by the mingw runtime startup code. I forgot the details of your analysis, but I think you are right. However, I would feel more comfortable if only a single CRT was used from an extension module. As for creating a case that crashes if you mix CRTs: Just try PyRun_SimpleFile in the extension, with a file that you fopen'ed in the extension. Satisfy this fopen from, say, msvcrt.dll, and load the extension into Python 2.4. Regards, Martin From martin at v.loewis.de Mon Dec 13 23:31:57 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Dec 13 23:31:58 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <wtvma0qw.fsf@python.net> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> <wtvma0qw.fsf@python.net> Message-ID: <41BE185D.6050607@v.loewis.de> Thomas Heller wrote: > And recently I played with bindings to OpenGL's glut32.dll - glut calls > exit() from internal C code. If linked with the wrong CRT, this will do > nothing instead of terminating the process. Interesting. Looking at the code of exit(), it is clear that the wrong atexit handlers will be called if you use the wrong CRT (i.e. only one set of atexit handlers will be called, whereas you might have multiple lists of atexit handlers); see atonexit.c. It is not so clear why you could possibly get out of exit() without actually exiting. The only possible code path I see in the msvcr71.dll is when mscoree.dll is in the address space, in which case the CRT will invoke CorExitProcess (!). In all other cases, it will invoke ExitProcess, which, to my understanding, will terminate the process unconditionally. Regards, Martin From adam.jtm30 at gmail.com Tue Dec 14 00:11:13 2004 From: adam.jtm30 at gmail.com (Adam Bark) Date: Tue Dec 14 00:11:17 2004 Subject: [Python-Dev] Re: The Other Py2.4 issue? In-Reply-To: <20041213110006.ED83E1E4032@bag.python.org> References: <20041213110006.ED83E1E4032@bag.python.org> Message-ID: <be4fbf92041213151155de169d@mail.gmail.com> Even though there would be no significant speed-up I would still be interested in a proper compiler just to make smaller exe's so I can give out my programs. I used Py2EXE for my last program that was tiny but with Tkinter and all the Tcl/Tk gubbins it was over 5Mb. Also would it perhaps be possible to have some sort of psyco program that can optimize the compiled code on the fly? From rsenra at acm.org Tue Dec 14 04:05:09 2004 From: rsenra at acm.org (Rodrigo Dias Arruda Senra) Date: Tue Dec 14 04:06:10 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <20041214010509.4f4ad3ec@Goku> [ Guido van Rossum <gvanrossum@gmail.com> ] ----------------------------------------------- | One thing that bugs me: the article says 3 or 4 times that Python is | slow, each time with a refutation ("but it's so flexible", "but it's | fast enough") but still, they sure seem to harp on the point. That reminds me of this joke: """Couple of guys find themselves being chased by a lion. The first one starts to strecth out preparing to run, the second one finds that ridicule. - No way you'll run faster than that lion. While, the other replies: - I don't need to run faster than the lion, I just need to run faster than you. """ I love to see Python getting faster and faster. Eventually (if...when PyPy succedes) faster-than-C ! But after over 4 years of making-ends-meet with Python, our Brazilian customers never complained of our Python solutions being slow. Perhaps because they were glad enough with projects delivered 5 to 10 times faster than usual. But, but to the joke: I believe Python must strive to run at least as fast as the crowd -- Java, Perl, Ruby, Lua, Boo, etc Maybe we could visit the language shootout sites, translate Python snipets to modern 2.4 idioms and rerun the test suites: http://www.bagley.org/~doug/shootout/ # classic http://shootout.alioth.debian.org/ # modern My 2 cents. best regards, Senra -- ,_ | ) Rodrigo Senra <rsenra |at| acm.org> |(______ ----------------------------------------------- _( (|__|] GPr Sistemas http://www.gpr.com.br _ | (|___|] IC - Unicamp http://www.ic.unicamp.br/~921234 ___ (|__|] L___(|_|] ----------------------------------------------- From zhangyue99 at tsinghua.org.cn Tue Dec 14 04:09:58 2004 From: zhangyue99 at tsinghua.org.cn (=?UTF-8?B?WmhhbmdZdWXlvKDlsrM=?=) Date: Tue Dec 14 04:10:14 2004 Subject: [Python-Dev] httplib timeout patch Message-ID: <41BE5986.9090904@tsinghua.org.cn> Hi, Here is a patch for httplib. I added a timeout value for httplib.HTTPConnection, please check. (diff - CVS 1.94) Zhang Yue 2004-12-14 -------------- next part -------------- Index: httplib.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/httplib.py,v retrieving revision 1.94 diff -r1.94 httplib.py 575c575 < def __init__(self, host, port=None, strict=None): --- > def __init__(self, host, port=None, strict=None, timeout=None): 580a581 > self.timeout = timeout 613a615,616 > if self.timeout: > self.sock.settimeout(self.timeout) From bac at OCF.Berkeley.EDU Tue Dec 14 08:38:25 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Dec 14 08:38:36 2004 Subject: [Python-Dev] httplib timeout patch In-Reply-To: <41BE5986.9090904@tsinghua.org.cn> References: <41BE5986.9090904@tsinghua.org.cn> Message-ID: <41BE9871.5080408@ocf.berkeley.edu> ZhangYue?? wrote: > Hi, > > Here is a patch for httplib. I added a timeout value for > httplib.HTTPConnection, please check. > Thanks for the patch, but this is the wrong place for it. Please create a new patch item on SourceForge at http://sourceforge.net/patch/?group_id=5470 . -Brett From martin at v.loewis.de Tue Dec 14 09:19:31 2004 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue Dec 14 09:19:32 2004 Subject: [Python-Dev] httplib timeout patch In-Reply-To: <41BE9871.5080408@ocf.berkeley.edu> References: <41BE5986.9090904@tsinghua.org.cn> <41BE9871.5080408@ocf.berkeley.edu> Message-ID: <41BEA213.7070901@v.loewis.de> Brett C. wrote: >> Here is a patch for httplib. I added a timeout value for >> httplib.HTTPConnection, please check. >> > > Thanks for the patch, but this is the wrong place for it. Please create > a new patch item on SourceForge at > http://sourceforge.net/patch/?group_id=5470 . In addition, also avoid using plain diffs - use context or unified diffs instead. Regards, Martin From p.f.moore at gmail.com Tue Dec 14 11:01:34 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Tue Dec 14 11:01:37 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <41BE150F.9050100@v.loewis.de> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> <41BE150F.9050100@v.loewis.de> Message-ID: <79990c6b04121402011d8965bd@mail.gmail.com> On Mon, 13 Dec 2004 23:17:51 +0100, Martin v. L?wis <martin@v.loewis.de> wrote: > I forgot the details of your analysis, but I think you are right. > However, I would feel more comfortable if only a single CRT was used > from an extension module. Agreed. But to some extent I'm equally uncomfortable with the idea that the mingw developers added support for -lmsvcr71 and botched it badly enough for things like this not to work :-) > As for creating a case that crashes if you mix CRTs: Just try > PyRun_SimpleFile in the extension, with a file that you fopen'ed > in the extension. Satisfy this fopen from, say, msvcrt.dll, and > load the extension into Python 2.4. Sorry, I wasn't specific enough. Certainly, if you create a FILE* from one CRT, then pass it to another CRT, this will fail. But my point is, if your extension uses fopen() then the link command generated by distutils will ensure that the symbol is satisfied from msvcr71.dll. So your scenario cannot occur. The only symbols which can be satisfied from msvcrt in a mingw-compiled extension (assuming that distutils is used, or the -lmsvcr71 flag is otherwise added) are those referenced in the mingw startup code, but not in user code. As the startup code will never reference Python APIs, CRT data should never be shared between incompatible CRTs. Your point about atexit (from another email) may be an issue. However, again we have the fact that user C code is *always* satisfied from msvcr71, so any exit handlers added in C or Python will be registered with the msvcr71 CRT, and so, as the main python.exe uses msvcr71, will be executed. I don't see any signs that the mingw runtime for a DLL uses atexit (this by experiment, I haven't time at the moment to review the mingw sources) and so I don't believe that the startup code has issues here, either. Thanks for your comments. Your support for the viability of building extensions using mingw is important to me, so if you still have any concerns, let me know and I will do my best to address them. Paul. From p.f.moore at gmail.com Tue Dec 14 11:13:15 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Tue Dec 14 11:13:17 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <20041214010509.4f4ad3ec@Goku> References: <ca471dc204120814186f5dc423@mail.gmail.com> <20041214010509.4f4ad3ec@Goku> Message-ID: <79990c6b04121402137a577a5b@mail.gmail.com> On Tue, 14 Dec 2004 01:05:09 -0200, Rodrigo Dias Arruda Senra <rsenra@acm.org> wrote: > But, but to the joke: I believe Python must strive to run at least as fast as the crowd -- > Java, Perl, Ruby, Lua, Boo, etc > > Maybe we could visit the language shootout sites, translate Python snipets to modern > 2.4 idioms and rerun the test suites: Bill Janssen <janssen@parc.com> wrote: > In the last couple of months, an acquaintance of mine has been trying > out Python. He likes the language, particularly the support for > character sets, but has (apparently) given up after benchmarking a > typical (for him) task against Perl. The task is to use the "re" > package to heavily modify a file by a sequence of re substitutions. That might be a good idiom to review/translate/test. It's not a particularly natural thing to do in Python (REs are used much less in Python), but it's *very* natural in Perl, and a naive Perl->Python translation will make Python look slower. So a case study showing how you might "really" do a task like this in Python would be a useful resource. Also, it might point to somewhere where there are opportunities for optimisation within Python (for some reason this reminds me of the mutable byte buffer discussions from a while back). Paul From amk at amk.ca Tue Dec 14 15:31:01 2004 From: amk at amk.ca (A.M. Kuchling) Date: Tue Dec 14 15:31:42 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <864d3709041212093274becde2@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> Message-ID: <20041214143101.GA1404@rogue.amk.ca> On Sun, Dec 12, 2004 at 03:32:03PM -0200, Carlos Ribeiro wrote: > Of course, the point here is not Perl-bashing. The point here is that > we should be able to "sell" Python better than we do now, even without > the need to resort to such poor measures. I'm sure the Python > community does have good & creative people that can write a good > "selling" FAQ for Python, emphasizing the main points of the language. No one disagrees that Python needs better marketing material. At the last PyCon a group of people sat down in a pydotorg BoF and agreed that yes, we do need a management-friendly marketing site, and that we could put it on a separate hostname (something.python.org) so that the current www.python.org wouldn't have to be changed. However, no one has actually sat down and written such a site, or even outlined it. Let me encourage you to go ahead and do that. You could draft the outline on a Wiki page, and then later figure out an attractive design and organization for a new site. --amk From martin at v.loewis.de Tue Dec 14 23:48:41 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Dec 14 23:48:41 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <79990c6b04121402011d8965bd@mail.gmail.com> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> <41BE150F.9050100@v.loewis.de> <79990c6b04121402011d8965bd@mail.gmail.com> Message-ID: <41BF6DC9.8030500@v.loewis.de> Paul Moore wrote: > Thanks for your comments. Your support for the viability of building > extensions using mingw is important to me, so if you still have any > concerns, let me know and I will do my best to address them. I understand that one still needs to build libpython24.a in order to use this process. As I have said, I'd happily ship that file with the 2.4.1 MSI, unless the release manager tells me that this would an unacceptable new feature, and as long as somebody provides a fully automatic build process integrated into msi.py; for that build process, it is ok to assume that a cygwin installation is in c:\cygwin. So if this would be useful (which I don't know for sure), I still need a volunteer to contribute the appropriate magic. Regards, Martin From tdelaney at avaya.com Wed Dec 15 00:50:23 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Wed Dec 15 00:50:28 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE721221@au3010avexu1.global.avaya.com> "Martin v. L?wis" wrote: > Paul Moore wrote: >> Thanks for your comments. Your support for the viability of building >> extensions using mingw is important to me, so if you still have any >> concerns, let me know and I will do my best to address them. > > I understand that one still needs to build libpython24.a in order to > use this process. As I have said, I'd happily ship that file with the > 2.4.1 MSI, unless the release manager tells me that this would an > unacceptable new feature, and as long as somebody provides a fully > automatic build process integrated into msi.py; for that build > process, > it is ok to assume that a cygwin installation is in c:\cygwin. I think we should aim to support MSYS as well as Cygwin, but perhaps not for the first version where this goes in. OTOH, is it really necessary to have either MSYS or Cygwin? MinGW32 works standalone - distutils should be able to just invoke it. Tim Delaney From martin at v.loewis.de Wed Dec 15 01:08:03 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Dec 15 01:08:01 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE721221@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721221@au3010avexu1.global.avaya.com> Message-ID: <41BF8063.4030602@v.loewis.de> Delaney, Timothy C (Timothy) wrote: >> it is ok to assume that a cygwin installation is in c:\cygwin. > > > I think we should aim to support MSYS as well as Cygwin, but perhaps > not for the first version where this goes in. Not at all. I'm talking about the release process, and prerequisites required in that process. This is worth mentioning because the list of prerequisites you need to perform a Python release is already quite long: - CVS - putty (for CVS) - VC7.1 - current, built versions of all libraries (zlib, bzip2, tcl/tk, bsddb, ...) - Perl (needed to build OpenSSL) - HTML help workshop - (the platform SDK)(to build Itanium binaries, if desired) Now, I would be willing to add Cygwin on top of that - whether the build process also works with MSYS does not matter to me; I would not like a build process that *required* MSYS as a matter of preference. > OTOH, is it really necessary to have either MSYS or Cygwin? MinGW32 > works standalone - distutils should be able to just invoke it. I'm not talking about distutils, here. Regards, Martin From tdelaney at avaya.com Wed Dec 15 01:50:01 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Wed Dec 15 01:50:13 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE721222@au3010avexu1.global.avaya.com> "Martin v. L?wis" wrote: > Not at all. I'm talking about the release process, and prerequisites > required in that process. This is worth mentioning because the list > of prerequisites you need to perform a Python release is already quite > long: Ah - sorry - misinterpreted. What is the size of the generated libpython24.a? Unfortunately, I don't happen to have binutils installed in my work cygwin (pulling it down now, but it's very slow ...) so I can't check myself (I've built the 2.3 one at home before ...). Tim Delaney From greg at electricrain.com Wed Dec 15 07:21:58 2004 From: greg at electricrain.com (Gregory P. Smith) Date: Wed Dec 15 07:22:02 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <20041214143101.GA1404@rogue.amk.ca> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> Message-ID: <20041215062158.GA12070@zot.electricrain.com> > > we should be able to "sell" Python better than we do now, even without > > the need to resort to such poor measures. I'm sure the Python > > community does have good & creative people that can write a good > > "selling" FAQ for Python, emphasizing the main points of the language. > > No one disagrees that Python needs better marketing material. At the > last PyCon a group of people sat down in a pydotorg BoF and agreed > that yes, we do need a management-friendly marketing site, and that we > could put it on a separate hostname (something.python.org) so that the > current www.python.org wouldn't have to be changed. > > However, no one has actually sat down and written such a site, or even > outlined it. Let me encourage you to go ahead and do that. You could > draft the outline on a Wiki page, and then later figure out an > attractive design and organization for a new site. suggested hostname: why.python.org From p.f.moore at gmail.com Wed Dec 15 10:49:18 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Wed Dec 15 10:49:20 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <41BF6DC9.8030500@v.loewis.de> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> <41BE150F.9050100@v.loewis.de> <79990c6b04121402011d8965bd@mail.gmail.com> <41BF6DC9.8030500@v.loewis.de> Message-ID: <79990c6b041215014944f31097@mail.gmail.com> On Tue, 14 Dec 2004 23:48:41 +0100, Martin v. L?wis <martin@v.loewis.de> wrote: > I understand that one still needs to build libpython24.a in order to > use this process. As I have said, I'd happily ship that file with the > 2.4.1 MSI, unless the release manager tells me that this would an > unacceptable new feature, and as long as somebody provides a fully > automatic build process integrated into msi.py; for that build process, > it is ok to assume that a cygwin installation is in c:\cygwin. > > So if this would be useful (which I don't know for sure), I still need > a volunteer to contribute the appropriate magic. I am willing to work on this, if you wouldn't mind me asking dumb questions about the build process :-) However, I don't have VS.NET, so if automation requires integration into the VS project stuff, I can't help. For a starter, what steps do you actually take to build a release? I assume that the first step is to build Python, by clicking on "build" in VS.NET. Once you have that, is there a single script you run? If there's a document describing the release process somewhere, feel free to point me at it. Thanks, Paul. From phd at mail2.phd.pp.ru Wed Dec 15 11:02:41 2004 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed Dec 15 11:02:44 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <20041215062158.GA12070@zot.electricrain.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> <20041215062158.GA12070@zot.electricrain.com> Message-ID: <20041215100241.GA8106@phd.pp.ru> On Tue, Dec 14, 2004 at 10:21:58PM -0800, Gregory P. Smith wrote: > suggested hostname: why.python.org It's only a matter of taste, probably, but that looks a bit ugly for my eyes. May be use.python.org? corp.python.org? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From aoshi at OCF.Berkeley.EDU Wed Dec 15 12:35:01 2004 From: aoshi at OCF.Berkeley.EDU (Randy Chung) Date: Wed Dec 15 12:35:08 2004 Subject: [Python-Dev] Python in education Message-ID: <41C02165.2010000@ocf.berkeley.edu> Hi everyone, I'm going to be leading a class on Python at the University of California, Berkeley next semester (starting in January). I'm interested in using actual bugs in Python as exercises for the class, the goal being 1) to give the students something "real" to work with, and 2) to (hopefully) contribute back anything we manage to put together to the Python dev team. What I'm looking for are some oustanding bugs which the more experienced devs feel would be straightforward to implement (which might not have been implemented yet because they're not very interesting, or there are simply too many other things to do right now). I'd appreciate any and all suggestions any of you might be able to provide. I think that Python will have a better chance at wider adoption if universities would provide courses for the language. I'm running this class partly because I'd like to give back to the community which produced a language I've enjoyed using, and any assistance would be greatly appreciated. If you feel this is off topic, please feel free to reply to me off-list. Thanks, Randy Chung From ncoghlan at iinet.net.au Wed Dec 15 13:32:39 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Dec 15 13:32:42 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <20041215100241.GA8106@phd.pp.ru> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> <20041215062158.GA12070@zot.electricrain.com> <20041215100241.GA8106@phd.pp.ru> Message-ID: <41C02EE7.3090308@iinet.net.au> Oleg Broytmann wrote: > On Tue, Dec 14, 2004 at 10:21:58PM -0800, Gregory P. Smith wrote: > >>suggested hostname: why.python.org > > > It's only a matter of taste, probably, but that looks a bit ugly for > my eyes. May be use.python.org? corp.python.org? about.python.org? And if someone ends up playing with the DNS server, maybe they could add wiki.python.org while they're at it :) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From barry at python.org Wed Dec 15 14:12:12 2004 From: barry at python.org (Barry Warsaw) Date: Wed Dec 15 14:12:17 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <41C02EE7.3090308@iinet.net.au> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> <20041215062158.GA12070@zot.electricrain.com> <20041215100241.GA8106@phd.pp.ru> <41C02EE7.3090308@iinet.net.au> Message-ID: <1103116332.9165.69.camel@geddy.wooz.org> On Wed, 2004-12-15 at 07:32, Nick Coghlan wrote: > about.python.org? > > And if someone ends up playing with the DNS server, maybe they could add > wiki.python.org while they're at it :) DNS changes have to go through pydotorg at python.org, since Thomas is the person currently able to add host names. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041215/2ef3889d/attachment.pgp From aahz at pythoncraft.com Wed Dec 15 16:28:24 2004 From: aahz at pythoncraft.com (Aahz) Date: Wed Dec 15 16:28:33 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <20041215062158.GA12070@zot.electricrain.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> <20041215062158.GA12070@zot.electricrain.com> Message-ID: <20041215152824.GA24746@panix.com> On Tue, Dec 14, 2004, Gregory P. Smith wrote: >Attribution deleted: >> >> No one disagrees that Python needs better marketing material. At the >> last PyCon a group of people sat down in a pydotorg BoF and agreed >> that yes, we do need a management-friendly marketing site, and that we >> could put it on a separate hostname (something.python.org) so that the >> current www.python.org wouldn't have to be changed. >> >> However, no one has actually sat down and written such a site, or even >> outlined it. Let me encourage you to go ahead and do that. You could >> draft the outline on a Wiki page, and then later figure out an >> attractive design and organization for a new site. > > suggested hostname: why.python.org This is where the process always gets bogged down. :-( Once we have material, that's the time to start arguing about where it should go. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From tjreedy at udel.edu Wed Dec 15 20:58:59 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Dec 15 20:58:55 2004 Subject: [Python-Dev] Re: Python in education References: <41C02165.2010000@ocf.berkeley.edu> Message-ID: <cpq51j$bq8$1@sea.gmane.org> "Randy Chung" <aoshi@OCF.Berkeley.EDU> wrote in message news:41C02165.2010000@ocf.berkeley.edu... > Hi everyone, > > I'm going to be leading a class on Python at the University of > California, Berkeley next semester (starting in January). Great. > I'm interested in using actual bugs in Python as exercises Please consider including review of existing patches. Besides being useful, it will also teach students how to submit good patches of their own. Your primary choice is whether to work on changes in the C code for the interpreter and builtin modules or changes in the Python code in standard library modules. The former would be more appropriate for a class on application programming in C, so I suggest the latter for learning Python programming. Besides which, improving the library will probably be a focus of the just-starting 2.5 cycle. You can access existing patch and bug lists via http://sourceforge.net/tracker/?group_id=5470 For each, you can narrow list by selecting, for instance, Category 'Python Library'. Terry J. Reedy From tjreedy at udel.edu Wed Dec 15 21:21:38 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Dec 15 21:21:48 2004 Subject: [Python-Dev] Re: Re: Re: 2.4 news reaches interesting places References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org><41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> <20041215062158.GA12070@zot.electricrain.com><20041215100241.GA8106@phd.pp.ru> <41C02EE7.3090308@iinet.net.au> Message-ID: <cpq6c2$gjr$1@sea.gmane.org> For a subsite aimed at businesses, business.python.org is obvious and easily remembered. Not all businesses are corporations. 'about' and 'why' are not specific at all. I think such a subsite, linked from the main site also, would be a good idea. It should explain both why (including success stories) and how (including employment/contracting) to use Python in business. Pieces could come from the existing site. I think the mere existence of such a site would constitute a message to businesses that Python is ready for business. I suspect the existence of even a skeletal site would attract contributions to build it further. Correspondingly, perhaps there should also be a science(or scitech or learning).python.org with pages on Astronomy, Bioscience, Chemistry, Mathematics, Education, etc. Perhaps the scipy and other science application groups would help. With two subsites for the main application areas, this would leave www.python.org to focus on the language itself. Terry J. Reedy From allison at sumeru.stanford.EDU Wed Dec 15 22:04:11 2004 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Wed Dec 15 22:04:17 2004 Subject: [Python-Dev] Re: Re: Re: 2.4 news reaches interesting places In-Reply-To: <cpq6c2$gjr$1@sea.gmane.org> Message-ID: <Pine.LNX.4.10.10412151302350.19316-100000@sumeru.stanford.EDU> Ummm... I don't think that anyone looks for information by trolling subdomain names. If I am looking for python information, I go to www.python.org or python.org. I would never guess "business.python.org". Seems to me that what we need is content and let the search engines bring on the masses. On Wed, 15 Dec 2004, Terry Reedy wrote: > For a subsite aimed at businesses, business.python.org is obvious and > easily remembered. Not all businesses are corporations. 'about' and 'why' > are not specific at all. > > I think such a subsite, linked from the main site also, would be a good > idea. It should explain both why (including success stories) and how > (including employment/contracting) to use Python in business. Pieces could > come from the existing site. > > I think the mere existence of such a site would constitute a message to > businesses that Python is ready for business. I suspect the existence of > even a skeletal site would attract contributions to build it further. > > Correspondingly, perhaps there should also be a > science(or scitech or learning).python.org with pages on Astronomy, > Bioscience, Chemistry, Mathematics, Education, etc. Perhaps the scipy and > other science application groups would help. > > With two subsites for the main application areas, this would leave > www.python.org to focus on the language itself. > > Terry J. Reedy > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40sumeru.stanford.edu > From martin at v.loewis.de Wed Dec 15 22:57:00 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Dec 15 22:56:57 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <79990c6b041215014944f31097@mail.gmail.com> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> <41BE150F.9050100@v.loewis.de> <79990c6b04121402011d8965bd@mail.gmail.com> <41BF6DC9.8030500@v.loewis.de> <79990c6b041215014944f31097@mail.gmail.com> Message-ID: <41C0B32C.7040701@v.loewis.de> Paul Moore wrote: > For a starter, what steps do you actually take to build a release? I > assume that the first step is to build Python, by clicking on "build" > in VS.NET. Yes. You can skip this step by just putting all the .pyds, dlls, and .exes into the PCbuild directory. The packaging will try to pick them up from there (and proceed if some are missing, like Tcl likely will). > Once you have that, is there a single script you run? Yes. Invoke Tools\msi\msi.py, using a Python that has pythonwin (i.e. COM interopability). The only tricky part is that you need cabarc.exe, which is included in VC, and in the platform SDK. > If > there's a document describing the release process somewhere, feel free > to point me at it. Tools/msi/README.txt Regards, Martin From bac at OCF.Berkeley.EDU Thu Dec 16 02:18:22 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Dec 16 02:18:28 2004 Subject: [Python-Dev] Re: Python in education In-Reply-To: <cpq51j$bq8$1@sea.gmane.org> References: <41C02165.2010000@ocf.berkeley.edu> <cpq51j$bq8$1@sea.gmane.org> Message-ID: <41C0E25E.1040606@ocf.berkeley.edu> Terry Reedy wrote: > "Randy Chung" <aoshi@OCF.Berkeley.EDU> wrote in message > news:41C02165.2010000@ocf.berkeley.edu... [SNIP] >> I'm interested in using actual bugs in Python as exercises > > > Please consider including review of existing patches. Besides being > useful, it will also teach students how to submit good patches of their > own. > To go along with this suggestion, see http://www.python.org/dev/dev_intro.html for the basic overview of what a review tends to consist of. -Brett From kdart at kdart.com Thu Dec 16 02:37:44 2004 From: kdart at kdart.com (Keith Dart) Date: Thu Dec 16 02:37:58 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <20041214143101.GA1404@rogue.amk.ca> References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> Message-ID: <41C0E6E8.2000805@kdart.com> A.M. Kuchling wrote: > On Sun, Dec 12, 2004 at 03:32:03PM -0200, Carlos Ribeiro wrote: > >>Of course, the point here is not Perl-bashing. The point here is that >>we should be able to "sell" Python better than we do now, even without >>the need to resort to such poor measures. I'm sure the Python >>community does have good & creative people that can write a good >>"selling" FAQ for Python, emphasizing the main points of the language. > > > No one disagrees that Python needs better marketing material. At the > last PyCon a group of people sat down in a pydotorg BoF and agreed > that yes, we do need a management-friendly marketing site, and that we > could put it on a separate hostname (something.python.org) so that the > current www.python.org wouldn't have to be changed. > > However, no one has actually sat down and written such a site, or even > outlined it. Let me encourage you to go ahead and do that. You could > draft the outline on a Wiki page, and then later figure out an > attractive design and organization for a new site. Whatever it looks like, it should probably run on Zope plus Plone. 8-) You know... eat your own dog food. 8-) The kind folks over at Zettai! have provided some space for me. Perhaps they will be glad to host the main Python site, as well? -- \/ \/ (O O) -- --------------------oOOo~(_)~oOOo---------------------------------------- Keith Dart <kdart@kdart.com> vcard: <http://www.kdart.com/~kdart/kdart.vcf> public key: ID: F3D288E4 URL: <http://www.kdart.com/~kdart/public.key> ============================================================================ From sdeibel at wingware.com Thu Dec 16 02:57:29 2004 From: sdeibel at wingware.com (Stephan Deibel) Date: Thu Dec 16 02:56:18 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <20041215152824.GA24746@panix.com> Message-ID: <Pine.LNX.4.50.0412152052030.3211-100000@hedgehog> On Wed, 15 Dec 2004, Aahz wrote: > On Tue, Dec 14, 2004, Gregory P. Smith wrote: > >Attribution deleted: > >> > >> No one disagrees that Python needs better marketing material. At the > >> last PyCon a group of people sat down in a pydotorg BoF and agreed > >> that yes, we do need a management-friendly marketing site, and that we > >> could put it on a separate hostname (something.python.org) so that the > >> current www.python.org wouldn't have to be changed. > >> > >> However, no one has actually sat down and written such a site, or even > >> outlined it. Let me encourage you to go ahead and do that. You could > >> draft the outline on a Wiki page, and then later figure out an > >> attractive design and organization for a new site. > > > > suggested hostname: why.python.org > > This is where the process always gets bogged down. :-( Once we have > material, that's the time to start arguing about where it should go. Absolutely. Content first, details later. Incidentally, I keep offering: Anyone that actually takes this on should feel free to rip off content from http://wingware.com/python -- I'm not saying this is the best content or the most complete but it's a start anyway (I wrote it in case that wasn't clear ;-) Incidentally, if someone does get excited about working on this, check with me or another PSF board member. - Stephan From johan at carlstedt.net Thu Dec 16 13:30:05 2004 From: johan at carlstedt.net (Johan Carlstedt) Date: Thu Dec 16 13:30:11 2004 Subject: [Python-Dev] Problems compiling Python 2.3.3 on Solaris 10 with gcc 3.4.1 Message-ID: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org> Hello, I am having problems compiling Python 2.3.3 on Solaris 10 X86 using gcc 3.4.1. The error message below is generated. I would be grateful for any advice. With friendly regards, Johan Carlstedt # make gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Modules/python.o Modules/python.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/acceler.o Parser/acceler.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/grammar1.o Parser/grammar1.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/listnode.o Parser/listnode.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/node.o Parser/node.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/parser.o Parser/parser.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/parsetok.o Parser/parsetok.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/bitset.o Parser/bitset.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/metagrammar.o Parser/metagrammar.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/firstsets.o Parser/firstsets.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/grammar.o Parser/grammar.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/pgen.o Parser/pgen.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/myreadline.o Parser/myreadline.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Parser/tokenizer.o Parser/tokenizer.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Objects/abstract.o Objects/abstract.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Objects/boolobject.o Objects/boolobject.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Objects/bufferobject.o Objects/bufferobject.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Objects/cellobject.o Objects/cellobject.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Objects/classobject.o Objects/classobject.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Objects/cobject.o Objects/cobject.c gcc -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. - I./Include -DPy_BUILD_CORE -o Objects/complexobject.o Objects/complexobject.c Objects/complexobject.c: In function `complex_pow': Objects/complexobject.c:469: error: invalid operands to binary == Objects/complexobject.c:469: error: wrong type argument to unary minus Objects/complexobject.c:469: error: invalid operands to binary == Objects/complexobject.c:469: error: wrong type argument to unary minus *** Error code 1 make: Fatal error: Command failed for target `Objects/complexobject.o' # gcc --version gcc (GCC) 3.4.1 Copyright (C) 2004 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # uname -a SunOS sunbuild1 5.10 s10_71 i86pc i386 i86pc From eirik.mikkelsen at unix.net Thu Dec 16 14:03:55 2004 From: eirik.mikkelsen at unix.net (Eirik Mikkelsen) Date: Thu Dec 16 14:02:32 2004 Subject: [Python-Dev] Problems compiling Python 2.3.3 on Solaris 10 with gcc 3.4.1 In-Reply-To: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org> References: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org> Message-ID: <1103202235.10436.9.camel@localhost> On Thu, 2004-12-16 at 12:30 +0000, Johan Carlstedt wrote: > Hello, > > I am having problems compiling Python 2.3.3 on Solaris 10 X86 using gcc > 3.4.1. The error message below is generated. > [SNIP] > Objects/complexobject.c > Objects/complexobject.c: In function `complex_pow': > Objects/complexobject.c:469: error: invalid operands to binary == > Objects/complexobject.c:469: error: wrong type argument to unary minus > Objects/complexobject.c:469: error: invalid operands to binary == > Objects/complexobject.c:469: error: wrong type argument to unary minus Solaris 10 doesn't correctly define HUGE_VAL (I think it's called HUGE in their header files) - so you may want to change the #define in Include/pyport.h from #define Py_HUGE_VAL HUGE_VAL to #define Py_HUGE_VAL HUGE -- Eirik Mikkelsen <eirik.mikkelsen@unix.net> From gvanrossum at gmail.com Thu Dec 16 17:15:43 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Dec 16 17:15:46 2004 Subject: [Python-Dev] Problems compiling Python 2.3.3 on Solaris 10 with gcc 3.4.1 In-Reply-To: <1103202235.10436.9.camel@localhost> References: <29507.147.114.226.181.1103200205.squirrel@www.flirble.org> <1103202235.10436.9.camel@localhost> Message-ID: <ca471dc204121608157f76ef65@mail.gmail.com> If this still occurs with Python 2.3.4 or 2.4, please file a bug so it can be fixed (python-dev isn't a good place to remember fixes). On Thu, 16 Dec 2004 14:03:55 +0100, Eirik Mikkelsen <eirik.mikkelsen@unix.net> wrote: > On Thu, 2004-12-16 at 12:30 +0000, Johan Carlstedt wrote: > > Hello, > > > > I am having problems compiling Python 2.3.3 on Solaris 10 X86 using gcc > > 3.4.1. The error message below is generated. > > > [SNIP] > > Objects/complexobject.c > > Objects/complexobject.c: In function `complex_pow': > > Objects/complexobject.c:469: error: invalid operands to binary == > > Objects/complexobject.c:469: error: wrong type argument to unary minus > > Objects/complexobject.c:469: error: invalid operands to binary == > > Objects/complexobject.c:469: error: wrong type argument to unary minus > > Solaris 10 doesn't correctly define HUGE_VAL (I think it's called HUGE > in their header files) - so you may want to change the #define in > Include/pyport.h from > > #define Py_HUGE_VAL HUGE_VAL > > to > > #define Py_HUGE_VAL HUGE > > -- > Eirik Mikkelsen <eirik.mikkelsen@unix.net> > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bsder at mail.allcaps.org Thu Dec 16 18:12:58 2004 From: bsder at mail.allcaps.org (Andrew P. Lentvorski, Jr.) Date: Thu Dec 16 18:13:09 2004 Subject: [Python-Dev] Re: Python in education In-Reply-To: <cpq51j$bq8$1@sea.gmane.org> References: <41C02165.2010000@ocf.berkeley.edu> <cpq51j$bq8$1@sea.gmane.org> Message-ID: <BBF201B5-4F85-11D9-B566-000A95C874EE@mail.allcaps.org> On Dec 15, 2004, at 2:58 PM, Terry Reedy wrote: > Your primary choice is whether to work on changes in the C code for the > interpreter and builtin modules or changes in the Python code in > standard > library modules. The PyPy folks might be useful too given that they are writing Python in Python. http://codespeak.net/pypy/ -a From stewart.midwinter at gmail.com Thu Dec 16 14:13:41 2004 From: stewart.midwinter at gmail.com (Stewart Midwinter) Date: Thu Dec 16 23:57:18 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places Message-ID: <d4c0d4800412160513d16f13e@mail.gmail.com> A number of people commented on the article in GCN, at http://gcn.com/vol1_no1/daily-updates/28026-1.html, and wondered if more could not be done to address the perception of speed. The point is made that although people mention all of the benefits of Python, like ease of use, flexibility, they always come back to making references to its speed. And the question is raised, "what can we do to counter this perception?". I think the answer lies in a quote by a user in that same article: "At first, Doak was worried a Python-based program would not run simulations quickly enough, however he found performance to be acceptable. Let's turn this around. Forget about trying to create a perception that Python is fast. Compiled languages will always be faster, at least for large applications. Or at least they'll be perceived that way. So let's acknowledge that upfront, but say "Python is fast enough for most uses", but then go on to say "and in addition to its acceptable speed, it offers many advantages like ease-of-use, flexibility, easy code maintenance (since the code is still understandable 6 months later!) etc. Marketers of other products have used this same technique successfully. For example, at one time there was a perception that Kellogg's Corn Flakes were old and boring. Sales were slipping. Rather than refute that, marketers turned the issue on its head by emphasizing that the product had been around a long time because it was good, and good for you. Hence was born the slogan "taste them again, for the first time". Possible slogan for Python: "Fast enough, and better in many ways". cheers S http://midtoad.homelinux.org/wp/ -- Stewart Midwinter stewart@midwinter.ca stewart.midwinter@gmail.com From michael.walter at gmail.com Fri Dec 17 02:14:25 2004 From: michael.walter at gmail.com (Michael Walter) Date: Fri Dec 17 02:14:27 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places In-Reply-To: <d4c0d4800412160513d16f13e@mail.gmail.com> References: <d4c0d4800412160513d16f13e@mail.gmail.com> Message-ID: <877e9a1704121617142070df06@mail.gmail.com> On Thu, 16 Dec 2004 06:13:41 -0700, Stewart Midwinter <stewart.midwinter@gmail.com> wrote: > [...] Possible slogan for Python: "Fast enough, and better in many ways". [...] Let's make it faster first :-) Cheers, Michael From carribeiro at gmail.com Fri Dec 17 12:20:18 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Fri Dec 17 12:20:21 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places In-Reply-To: <d4c0d4800412160513d16f13e@mail.gmail.com> References: <d4c0d4800412160513d16f13e@mail.gmail.com> Message-ID: <864d37090412170320561ecb8@mail.gmail.com> On Thu, 16 Dec 2004 06:13:41 -0700, Stewart Midwinter <stewart.midwinter@gmail.com> wrote: > A number of people commented on the article in GCN, at > http://gcn.com/vol1_no1/daily-updates/28026-1.html, and wondered if > more could not be done to address the perception of speed. The point > is made that although people mention all of the benefits of Python, > like ease of use, flexibility, they always come back to making > references to its speed. And the question is raised, "what can we do > to counter this perception?". I think the answer lies in a quote by a > user in that same article: "At first, Doak was worried a Python-based > program would not run simulations quickly enough, however he found > performance to be acceptable. > > Let's turn this around. Forget about trying to create a perception > that Python is fast. Compiled languages will always be faster, at > least for large applications. Or at least they'll be perceived that > way. So let's acknowledge that upfront, but say "Python is fast > enough for most uses", but then go on to say "and in addition to its > acceptable speed, it offers many advantages like ease-of-use, > flexibility, easy code maintenance (since the code is still > understandable 6 months later!) etc. > > Marketers of other products have used this same technique > successfully. For example, at one time there was a perception that > Kellogg's Corn Flakes were old and boring. Sales were slipping. > Rather than refute that, marketers turned the issue on its head by > emphasizing that the product had been around a long time because it > was good, and good for you. Hence was born the slogan "taste them > again, for the first time". > > Possible slogan for Python: "Fast enough, and better in many ways". One possible marketing strategy is to use the adjective "fast" in a broader sense. The Python slogan could be something like: "Programming has never been any faster" -- this changes the playing ground, from raw performance to *programming* performance. And sure, nothing beats Python (the overall package) in this respect. It can deliver fast code in a short time. Othere languages are faster to run, but take longer to code... BTW, I would move away from the "fast enough" when talking about performance. It's difficult to qualify what is "enough" in marketing terms; also, a selling/winning message can't be seen as taking excuses for any reason. On the other hand, Python never claims to be the fastest language on raw execution performance, but only to be fast; but in this sense, being "fast enough" is the same as being "fast". So, I would never say, "Python allows you to write fast enough code in a short time"; I would say "Python allows you to write fast code in a short time". Leave the "fast enough" out of this, please. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From python at rcn.com Fri Dec 17 13:47:19 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Dec 17 13:50:20 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules posixmodule.c, 2.300.8.10, 2.300.8.11 In-Reply-To: <E1CdPE6-0003N9-VL@sc8-pr-cvs1.sourceforge.net> Message-ID: <000401c4e436$8eabb0c0$e841fea9@oemcomputer> The Dec 12th check-ins break tests on WinME: test_glob.py ------------ Traceback (most recent call last): File "test_glob.py", line 78, in test_glob_one_directory eq(self.glob('a*'), map(self.norm, ['a', 'aab', 'aaa'])) File "test_glob.py", line 67, in assertSequencesEqual_noorder self.assertEqual(set(l1), set(l2)) AssertionError: set(['/tmp\\@test_dir\\aaa', '/tmp\\@test_dir\\aab', '/tmp\\@tes t_dir\\a']) != set(['\\tmp\\@test_dir\\aab', '\\tmp\\@test_dir\\aaa', '\\tmp\\@t est_dir\\a']) test_urllib.py -------------- FAIL: test_basic (__main__.urlretrieve_FileTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_urllib.py", line 142, in test_basic self.assertEqual(result[0], test_support.TESTFN) AssertionError: '\\tmp\\@test' != '/tmp\\@test' Please fix, Raymond > -----Original Message----- > From: python-checkins-bounces@python.org [mailto:python-checkins- > bounces@python.org] On Behalf Of aimacintyre@users.sourceforge.net > Sent: Sunday, December 12, 2004 3:37 AM > To: python-checkins@python.org > Subject: [Python-checkins] python/dist/src/Modules posixmodule.c, > 2.300.8.10, 2.300.8.11 > > Update of /cvsroot/python/python/dist/src/Modules > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12893 > > Modified Files: > Tag: release23-maint > posixmodule.c > Log Message: > OS/2 specific fixes related to SF bug # 1003471. > > Also revise a related function to minimise file handle/pipe leakage > and improve reliability. > > Backported from -HEAD. From mwh at python.net Fri Dec 17 14:49:45 2004 From: mwh at python.net (Michael Hudson) Date: Fri Dec 17 14:49:47 2004 Subject: [Python-Dev] Re: Re: 2.4 news reaches interesting places In-Reply-To: <41C0E6E8.2000805@kdart.com> (Keith Dart's message of "Wed, 15 Dec 2004 17:37:44 -0800") References: <ca471dc204120814186f5dc423@mail.gmail.com> <cpeh62$ldh$1@sea.gmane.org> <ca471dc204121108225511d43@mail.gmail.com> <cpfcqh$ejc$1@sea.gmane.org> <41BC6D92.5050802@heneryd.com> <864d370904121209147b713f31@mail.gmail.com> <864d3709041212093274becde2@mail.gmail.com> <20041214143101.GA1404@rogue.amk.ca> <41C0E6E8.2000805@kdart.com> Message-ID: <2mr7lpkpae.fsf@starship.python.net> Keith Dart <kdart@kdart.com> writes: > A.M. Kuchling wrote: >> On Sun, Dec 12, 2004 at 03:32:03PM -0200, Carlos Ribeiro wrote: >> >>>Of course, the point here is not Perl-bashing. The point here is that >>>we should be able to "sell" Python better than we do now, even without >>>the need to resort to such poor measures. I'm sure the Python >>>community does have good & creative people that can write a good >>>"selling" FAQ for Python, emphasizing the main points of the language. >> No one disagrees that Python needs better marketing material. At >> the >> last PyCon a group of people sat down in a pydotorg BoF and agreed >> that yes, we do need a management-friendly marketing site, and that we >> could put it on a separate hostname (something.python.org) so that the >> current www.python.org wouldn't have to be changed. >> However, no one has actually sat down and written such a site, or >> even >> outlined it. Let me encourage you to go ahead and do that. You could >> draft the outline on a Wiki page, and then later figure out an >> attractive design and organization for a new site. > > > Whatever it looks like, it should probably run on Zope plus Plone. 8-) > You know... eat your own dog food. 8-) This is SO not the problem that needs discussion... > The kind folks over at Zettai! have provided some space for > me. Perhaps they will be glad to host the main Python site, as well? Neither is this! Cheers, mwh -- Just point your web browser at http://www.python.org/search/ and look for "program", "doesn't", "work", or "my". Whenever you find someone else whose program didn't work, don't do what they did. Repeat as needed. -- Tim Peters, on python-help, 16 Jun 1998 From mcherm at mcherm.com Fri Dec 17 17:24:11 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Fri Dec 17 17:24:12 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places Message-ID: <1103300651.41c3082b5f3d8@mcherm.com> Carlos Riberio writes: > One possible marketing strategy is to use the adjective "fast" in a > broader sense. The Python slogan could be something like: "Programming > has never been any faster" -- this changes the playing ground, from > raw performance to *programming* performance. I think Carlos is onto something here. Guido's original question was how to fight this meme... in other words, people think of Python as slow, whether they have measured it or not. Just like they think of Java as being more portable. Talking about "fast enough" is just another way of reminding people that we're really quite slow (even if that's not true). So how about a slogan like "Code it Fast, with Python", or "Python: Code Fast" -- one which emphasizes the (easily defended) claim that development time is shorter with Python, but which at the same time manages to associate the word "fast" with "Python". -- Michael Chermside From python at rcn.com Fri Dec 17 18:06:34 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Dec 17 18:09:34 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places In-Reply-To: <1103300651.41c3082b5f3d8@mcherm.com> Message-ID: <001801c4e45a$c3979fa0$e841fea9@oemcomputer> > So how about a slogan like "Code it Fast, with Python", or "Python: Code > Fast" -- one which emphasizes the (easily defended) claim that development > time is shorter with Python, but which at the same time manages to > associate the word "fast" with "Python". I always liked: "Python, the language that wraps tightly around a problem and swallows it whole." Raymond From FBatista at uniFON.com.ar Fri Dec 17 18:21:54 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Fri Dec 17 18:24:52 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places Message-ID: <A128D751272CD411BC9200508BC2194D053C7D31@escpl.tcp.com.ar> [Raymond Hettinger] #- > So how about a slogan like "Code it Fast, with Python", or "Python: #- Code #- > Fast" -- one which emphasizes the (easily defended) claim that #- development #- > time is shorter with Python, but which at the same time manages to #- > associate the word "fast" with "Python". #- #- I always liked: "Python, the language that wraps tightly around a #- problem and swallows it whole." Yes, but the point is that, even if it swallows the problem not-as-fast as C, it wraps around the problem very quickly! . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA. La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20041217/ed113b44/attachment.html From carribeiro at gmail.com Fri Dec 17 19:37:05 2004 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Fri Dec 17 19:37:08 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places In-Reply-To: <1103300651.41c3082b5f3d8@mcherm.com> References: <1103300651.41c3082b5f3d8@mcherm.com> Message-ID: <864d37090412171037364834f3@mail.gmail.com> On Fri, 17 Dec 2004 08:24:11 -0800, Michael Chermside <mcherm@mcherm.com> wrote: > Carlos Riberio writes: > > One possible marketing strategy is to use the adjective "fast" in a > > broader sense. The Python slogan could be something like: "Programming > > has never been any faster" -- this changes the playing ground, from > > raw performance to *programming* performance. > > I think Carlos is onto something here. Guido's original question was > how to fight this meme... in other words, people think of Python as > slow, whether they have measured it or not. Just like they think of > Java as being more portable. Talking about "fast enough" is just another > way of reminding people that we're really quite slow (even if that's not > true). > > So how about a slogan like "Code it Fast, with Python", or "Python: Code > Fast" -- one which emphasizes the (easily defended) claim that development > time is shorter with Python, but which at the same time manages to > associate the word "fast" with "Python". Whatever the slogan is, the most important point is: Python (and Pythonistas in general) should not be ashamed about Python's speed. I've checked most of the links that popped up as this thread progressed, and followed other similar links. In many cases, Python advocates are quoted as saying that speed improvements were due to recoding in C. That's definitely *not* the way we're going to sell the perception of speed. It is an unnecessary apology, and it sells the wrong idea. It says "we know it's slow, but we're moving to a faster language to improve performance". Not exactly reassuring. As far as the slogan is concerned - I still stand for my proposal, but I don't know if anyone has registered it first... I fear it is, it sounds strangely familiar. But I hope not. p.s. Can someone imagine a Microsoft senior executive saying that "C# is now faster because we rewrote the critical parts in C?" Or the same for Sun & Java? ;-) -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From FBatista at uniFON.com.ar Fri Dec 17 19:38:55 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Fri Dec 17 19:41:55 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places Message-ID: <A128D751272CD411BC9200508BC2194D053C7D37@escpl.tcp.com.ar> [Carlos Ribeiro] #- As far as the slogan is concerned - I still stand for my #- proposal, but #- I don't know if anyone has registered it first... I fear it is, it #- sounds strangely familiar. But I hope not. Well, Google didn't return any match to the exact phrase "Programming has never been any faster" (but this is not legal to affirm prior invention (yet)). I like your proposal (and slogan) a LOT! . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA. La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20041217/6dc675ac/attachment.html From eirik.mikkelsen at unix.net Fri Dec 17 23:05:08 2004 From: eirik.mikkelsen at unix.net (Eirik Mikkelsen) Date: Fri Dec 17 23:03:52 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places In-Reply-To: <864d37090412170320561ecb8@mail.gmail.com> References: <d4c0d4800412160513d16f13e@mail.gmail.com> <864d37090412170320561ecb8@mail.gmail.com> Message-ID: <1103321109.10602.28.camel@localhost> On Fri, 2004-12-17 at 09:20 -0200, Carlos Ribeiro wrote: > BTW, I would move away from the "fast enough" when talking about > performance. It's difficult to qualify what is "enough" in marketing > terms; also, a selling/winning message can't be seen as taking excuses > for any reason. On the other hand, Python never claims to be the > fastest language on raw execution performance, but only to be fast; > but in this sense, being "fast enough" is the same as being "fast". > So, I would never say, "Python allows you to write fast enough code in > a short time"; I would say "Python allows you to write fast code in a > short time". Leave the "fast enough" out of this, please. I totally agree. Personally, the first thing I think of when I see "enough" is "640k aught to be enough for anybody" (quote from you-know-who), like you are defining the needs of the user. Promote the strong sides, don't excuse the weak ones. "Fast enough" is not a positive marketing term, it's an excuse for a problem which I fail to see in Python. Hardware cost is way lower than programmer cost - I am convinced that in most cases the total expence is lower for a Python solution compared to an equal performing C, C++, C# or Java solution. -- Eirik Mikkelsen <eirik.mikkelsen@unix.net> From andymac at bullseye.apana.org.au Fri Dec 17 23:53:17 2004 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Sat Dec 18 01:11:55 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules posixmodule.c, 2.300.8.10, 2.300.8.11 In-Reply-To: <000401c4e436$8eabb0c0$e841fea9@oemcomputer> References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer> Message-ID: <20041218091927.E1961@bullseye.apana.org.au> On Fri, 17 Dec 2004, Raymond Hettinger wrote: > The Dec 12th check-ins break tests on WinME: > > > test_glob.py > ------------ > Traceback (most recent call last): > File "test_glob.py", line 78, in test_glob_one_directory > eq(self.glob('a*'), map(self.norm, ['a', 'aab', 'aaa'])) > File "test_glob.py", line 67, in assertSequencesEqual_noorder > self.assertEqual(set(l1), set(l2)) > AssertionError: set(['/tmp\\@test_dir\\aaa', '/tmp\\@test_dir\\aab', > '/tmp\\@tes > t_dir\\a']) != set(['\\tmp\\@test_dir\\aab', '\\tmp\\@test_dir\\aaa', > '\\tmp\\@t > est_dir\\a']) > > > test_urllib.py > -------------- > FAIL: test_basic (__main__.urlretrieve_FileTests) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "test_urllib.py", line 142, in test_basic > self.assertEqual(result[0], test_support.TESTFN) > AssertionError: '\\tmp\\@test' != '/tmp\\@test' > > > > Please fix, ???? I don't see any possible way for those checkins to affect any platform other than OS/2. 2 of the files are platform specific files (PC/os2emx/getpath.c, PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is contained within a platform specific #if/#endif: ... #ifdef HAVE_POPEN <<<-- line 3241 ... #if defined(PYOS_OS2) #if defined(PYCC_VACPP) ... <<<-- changes here #elif defined(PYCC_GCC) ... #endif /* PYCC_??? */ #elif defined(MS_WINDOWS) ... #else /* which OS? */ ... #endif /* PYOS_??? */ #endif /* HAVE_POPEN */ ... Note that the posixmodule change affects popen(). The matching PC/getpath.c changes that apply to Windows were checked in by Tim Peters back in August. I don't have any Windows development environment, so can't cross check your report :-( If you have verified the cause of failure by backing out v2.300.8.11, then there's a deeper problem about the Windows build picking up explicitly non-Windows components - which I find hard to believe. In the absence of more evidence, not guilty your honour. ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From andymac at bullseye.apana.org.au Sat Dec 18 00:10:35 2004 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Sat Dec 18 01:11:57 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places In-Reply-To: <001801c4e45a$c3979fa0$e841fea9@oemcomputer> References: <001801c4e45a$c3979fa0$e841fea9@oemcomputer> Message-ID: <20041218100407.F1961@bullseye.apana.org.au> On Fri, 17 Dec 2004, Raymond Hettinger wrote: > > So how about a slogan like "Code it Fast, with Python", or "Python: > Code > > Fast" -- one which emphasizes the (easily defended) claim that > development > > time is shorter with Python, but which at the same time manages to > > associate the word "fast" with "Python". > > I always liked: "Python, the language that wraps tightly around a > problem and swallows it whole." The only problem here is the association with the reptilian python, which is not perceived as being rapid - though it is highly effective as you describe. Whereas cobras... I somewhat suspect a subliminal association linking the reptile and the language, colouring perceptions without factual basis... :-( ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From kbk at shore.net Sat Dec 18 04:21:15 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Dec 18 04:21:37 2004 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200412180321.iBI3LFlA022448@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 259 open ( +0) / 2707 closed ( +2) / 2966 total ( +2) Bugs : 822 open (+22) / 4685 closed (+23) / 5507 total (+45) RFE : 160 open ( +0) / 139 closed ( +2) / 299 total ( +2) New / Reopened Patches ______________________ repair typo (2004-12-08) CLOSED http://python.org/sf/1080684 opened by George Yoshida Description of message_set and command option to store() (2004-12-13) http://python.org/sf/1084092 opened by Sean Reifschneider Patches Closed ______________ repair typo (2004-12-07) http://python.org/sf/1080684 closed by jlgijsbers fix inspect.getsource breaking with line-continuation & more (2004-08-19) http://python.org/sf/1011890 closed by jlgijsbers New / Reopened Bugs ___________________ Inplace set merge produces wrong results (2004-12-07) CLOSED http://python.org/sf/1080424 opened by Matthias Klose float issue for NaN type in .pyc file (2004-12-07) http://python.org/sf/1080440 opened by Dileep Nirala Python Icon in system Tray (2004-12-07) CLOSED http://python.org/sf/1080634 opened by Dileep Nirala thread.error: release unlocked lock on Queue put (2004-12-07) CLOSED http://python.org/sf/1080660 opened by John Speno os.ttyname() accepts wrong arguments (2004-12-07) http://python.org/sf/1080713 opened by Christian H?ltje full test with all unicode text files (2004-12-07) CLOSED http://python.org/sf/1080811 opened by Grzegorz Makarewicz locale.py doesn't recognize valid locale setting (2004-12-07) CLOSED http://python.org/sf/1080864 opened by stas Z readline module doesn't build on MacOSX (2004-12-07) http://python.org/sf/1081045 opened by Skip Montanaro Bad reference in whrandom docs (2004-12-08) CLOSED http://python.org/sf/1081370 opened by Lars Marius Garshol LDAP search segfaults on RedHat FC3 (2004-12-08) CLOSED http://python.org/sf/1081633 opened by Tessa Lau Rewrite of docs for compiler.visitor (2004-12-09) http://python.org/sf/1081824 opened by Kent Johnson Pydoc can't find browser (bug+solution!) (2004-12-08) http://python.org/sf/1081879 opened by Stewart Midwinter PyString_AsString() segfaults when passed a unicode string (2004-12-09) CLOSED http://python.org/sf/1082085 opened by Andreas Jung Documentation for PyUnicode_TailMatch incorrrectly says it r (2004-12-10) CLOSED http://python.org/sf/1082944 opened by Jim Fulton truncated gzip file triggers zlibmodule segfault (2004-12-10) http://python.org/sf/1083110 opened by Sam Rushing Change in signal function in the signal module (2004-12-10) http://python.org/sf/1083177 opened by Gary H. Loechelt Unable to see Python binary (2004-12-10) http://python.org/sf/1082874 opened by Prabal Rakshit UnboundLocalError raised by atexit module (2004-12-10) CLOSED http://python.org/sf/1083202 opened by Gary H. Loechelt Distutils doesn't pick up all the files it should. (2004-12-10) http://python.org/sf/1083299 opened by Mike Meyer font lock keyword regular expressions (2004-12-09) http://python.org/sf/1082487 opened by Robert Brown Tests fail instead of skip (2004-12-12) http://python.org/sf/1083645 opened by Detlef Vollmann Python 2.4 crashes (2004-12-12) http://python.org/sf/1083793 opened by Axel Kaiser functions replaced should point to its docs (2004-12-12) http://python.org/sf/1083895 opened by Johannes Gijsbers status of small floats in xml-rpc ? (2004-12-13) CLOSED http://python.org/sf/1084279 opened by Antoine Pitrou ossaudiodev no longer undocumented (2004-12-13) CLOSED http://python.org/sf/1084457 opened by Gregory H. Ball sys.stdin segfaults on invalid stdin (2004-12-13) http://python.org/sf/1084766 opened by Mihai Ibanescu list initialization bug (2004-12-14) CLOSED http://python.org/sf/1084906 opened by py_py gethostbyaddr on redhat for multiple hostnames (2004-12-14) http://python.org/sf/1085069 opened by Dave Kirby Fix stale link in PEP (2004-12-14) CLOSED http://python.org/sf/1085096 opened by Michael Chermside Uninstaller unclear about the app it's uninstalling (2004-12-14) CLOSED http://python.org/sf/1085168 opened by Giles Antonio Radford Uninstaller should restore file associations if possible (2004-12-14) http://python.org/sf/1085172 opened by Giles Antonio Radford Installation ends prematurely (2004-12-14) http://python.org/sf/1085208 opened by anamh0001 binascii.b2a_qp oddities (2004-12-14) http://python.org/sf/1085283 opened by DSM Mac Library Modules 1.1.1 Bad Info (2004-12-14) http://python.org/sf/1085300 opened by Walrus Make array.array pickle-able (2004-12-14) CLOSED http://python.org/sf/1085304 opened by Nicolas Fleury Bad interaction between PySequence_Fast and itertools (2004-12-15) CLOSED http://python.org/sf/1085744 opened by Nick Coghlan subprocess.Popen feature request (2004-12-15) http://python.org/sf/1085861 opened by Michele Simionato PySequence_Tuple not as fast as PySequence_List (2004-12-15) http://python.org/sf/1085744 reopened by rhettinger two strings holding the same value have different id (2004-12-15) CLOSED http://python.org/sf/1086096 opened by Stefan Seefeld Typo in module os documentation (2004-12-15) CLOSED http://python.org/sf/1086127 opened by Connelly refcount problem in syslog (2004-12-16) CLOSED http://python.org/sf/1086555 opened by DSM segfault in readline (2004-12-16) http://python.org/sf/1086603 opened by DSM Compile of _socket fails on 2.4 (2004-12-16) http://python.org/sf/1086642 opened by A. Stocker optparse docs missing section on Extending (2004-12-16) http://python.org/sf/1086675 opened by Kent Johnson "slots" member variable in object.h confuses Qt moc utility (2004-12-17) http://python.org/sf/1086854 opened by Jeremy Friesner datetime module documentation missing critical detail (2004-12-17) http://python.org/sf/1087216 opened by Mike Meyer Bugs Closed ___________ Inplace set merge produces wrong results (2004-12-07) http://python.org/sf/1080424 closed by tim_one thread.error: release unlocked lock on Queue put (2004-12-07) http://python.org/sf/1080660 closed by rhettinger full test with all unicode text files (2004-12-07) http://python.org/sf/1080811 closed by makaron locale.py doesn't recognize valid locale setting (2004-12-07) http://python.org/sf/1080864 closed by lemburg Bad reference in whrandom docs (2004-12-08) http://python.org/sf/1081370 closed by rhettinger PythonWin Tray Icon in system Tray on Windows platform (2004-12-07) http://python.org/sf/1080634 closed by loewis LDAP search segfaults on RedHat FC3 (2004-12-08) http://python.org/sf/1081633 closed by loewis PyString_AsString() segfaults when passed a unicode string (2004-12-09) http://python.org/sf/1082085 closed by dcjim Documentation for PyUnicode_TailMatch incorrrectly says it r (2004-12-10) http://python.org/sf/1082944 closed by rhettinger UnboundLocalError raised by atexit module (2004-12-10) http://python.org/sf/1083202 closed by rhettinger Python 1.5.2 security vulnerability still present in 2.3.4 (2004-08-05) http://python.org/sf/1003471 closed by aimacintyre test_subprocess fails on cygwin (2004-11-22) http://python.org/sf/1071516 closed by jlt63 Error building _bsddb extension (2004-12-01) http://python.org/sf/1077040 closed by greg status of small floats in xml-rpc ? (2004-12-13) http://python.org/sf/1084279 closed by rhettinger ossaudiodev no longer undocumented (2004-12-13) http://python.org/sf/1084457 closed by rhettinger list initialization bug (2004-12-14) http://python.org/sf/1084906 closed by rhettinger Fix stale link in PEP (2004-12-14) http://python.org/sf/1085096 closed by jlgijsbers Uninstaller unclear about the app it's uninstalling (2004-12-14) http://python.org/sf/1085168 closed by loewis Bad interaction between PySequence_Fast and itertools (2004-12-15) http://python.org/sf/1085744 closed by rhettinger Signal reception problem. FreeBSD 4.8 and 4.9 after forkpty (2004-04-28) http://python.org/sf/944119 closed by levis501 PySequence_Tuple not as fast as PySequence_List (2004-12-15) http://python.org/sf/1085744 closed by rhettinger two strings holding the same value have different id (2004-12-15) http://python.org/sf/1086096 closed by goodger Typo in module os documentation (2004-12-16) http://python.org/sf/1086127 closed by doerwalter refcount problem in syslog (2004-12-16) http://python.org/sf/1086555 closed by rhettinger New / Reopened RFE __________________ Add encoding to DocFileSuite (2004-12-07) http://python.org/sf/1080727 opened by Bjorn Tillenius RFE Closed __________ ignore element format character for PyArg_ParseTuple (2004-11-30) http://python.org/sf/1075902 closed by rhettinger Make array.array pickle-able (2004-12-14) http://python.org/sf/1085304 closed by rhettinger From bac at OCF.Berkeley.EDU Sat Dec 18 06:10:21 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Dec 18 06:10:27 2004 Subject: [Python-Dev] python-dev Summary for 2004-11-01 through 2004-11-15 [draft] Message-ID: <41C3BBBD.9050805@ocf.berkeley.edu> Now I am only a month behind. Won't send this out any earlier than Monday night, but probably won't be until Tuesday. ------------------------------------ ===================== Summary Announcements ===================== I am hoping to be caught up with my summary backlog by the end of the month. Hear is to hoping. ========= Summaries ========= ------------ 2.4 released ------------ `Python 2.4`_ has been released! Read the `What's New`_ for 2.4 to see what the major changes are since 2.3 . .. _Python 2.4: http://www.python.org/2.4/ .. _What's New: http://www.python.org/doc/2.4/whatsnew/whatsnew24.html Contributed threads: - `TRUNK FROZEN for 2.4b2 release from 0000UTC 3rd November (tomorrrow) <>`__ - `RELEASED Python 2.4, beta 2 <>`__ - `TRUNK UNFROZEN <>`__ ------------------------------------- Code coverage of the regression tests ------------------------------------- Walter D?rwald has a code coverage tool that runs once a month on Python's regression test at http://coverage.livinglogic.de/ . There was some issue with the 'trace' module having partially broken in 2.4 . The coverage was then run using ``Lib/test/regrtest.py -T`` to get more accurate numbers. Contributing threads: - `Code coverage tool updated <>`__ ------------------------------------ Garbage collecting weakref callbacks ------------------------------------ Tim Peters discovered that in 2.4 if a weakref's callback is still reachable but the referent and weakref itself are unreachable (Tim's example had an instance that contained a weakref to itself) that the callback is not called during garbage collection. This seemed inconsistent since if the weakref in Tim's example was replaced with an instance that contained a __del__ method that the method would have been called. Tim would like to clean this up so as to be able to ditch __del__ methods in Python 3000. The idea is that one would register a weakref with a callback for an object in itself that would be called when it is garbage collected. This would also negate the need for gc.garbage . Contributing threads: - `weakref gc semantics <>`__ - `patch-finalizer vs Zope3 <>`__ --------- PEP Watch --------- `PEP 336`_ introduces the idea of having None be a callable object. The result that calling None would return None itself. .. _PEP 336: http://www.python.org/peps/pep-0336.html Contributing threads: - `PEP 336, "Make None Callable", by Andrew McClelland <>`__ ---------------------------------------------- Discussion of including pysqlite in the stdlib ---------------------------------------------- The idea of including pysqlite_ in the stdlib came up once again (not this is the wrapper for SQLite_ and not SQLite itself). The arguments for including the module were the SQLite is most likely used more than Sleepycat and thus deserved a place in the stdlib if bsddb did. It also goes along with Python's idea of "batteries included". Arguments against started with the idea of sanctioning pysqlite over other SQLite wrappers did not seem proper. People also thought that including bsddb might not be correct anymore and thus should not be used as a basis for including pysqlite. This all then led into a discussion about package management and how to simplify extending what can be installed at compile-time. The idea of pushing PyPI_ came up as well as revising `PEP 206`_. .. _pysqlite: http://pysqlite.org/ .. _SQLite: http://www.sqlite.org/ .. _PyPI: http://www.python.org/pypi .. _PEP 206: http://www.python.org/peps/pep-0206.html Contributing threads: - `SQLite module for Python 2.5 <>`__ -------------------------------- 2.3 maintenance coming to an end -------------------------------- With Python 2.4 out, maintaining 2.3 is no longer a priority. Anthony Baxter, our beloved release manager, has said that 2.3.5 will be most likely coming out in January. After that, though, don't count on another 2.3 release since 2.4 will take its place as the version to maintain. All of this came about by the fact that Facundo Batista asked if closing 2.1 bugs was okay. As long as they have been fixed in the earliest version being maintained, closing them is fine. This goes for 2.2 as well. And as soon as 2.3.5 is out the door will most likely apply to 2.3 bugs. Contributing threads: - `Policy about old Python versions <>`__ ----------------------------------------------------------------- Sync/async servers in the stdlib and whether they should be there ----------------------------------------------------------------- There are multiple servers in the core for handling communications in both a synchronous and asynchronous manner. The idea came up of developing an API that all servers could follow in the core. While the discussion went back and forth with multiple mentions of PEAK, Twisted, and greenlets_, in the end the trend of the discussion suggested doing this well would be very difficult and not worth the effort. asyncore and asynchat also seemed to be deemed worth deprecation and thus not worth using. .. _PEAK: http://peak.telecommunity.com/ .. _Twisted: http://www.twistedmatrix.com/ .. _greenlets: http://codespeak.net/svn/user/arigo/greenlet/ Contributing threads: - `Synchronous and Asynchronous servers in the standard library <>`__ -------------------------------------------------- Getting the opcode count for rough profile numbers -------------------------------------------------- The idea was proposed of introducing sys.gettickeraccumulation which would contain the number of opcodes executed since the interpreter started. The idea is that you can get a rough estimate of how much time is being spent in Python code. The value of this has wavered back and forth with no clear answer. Contributing threads: - `proposal+patch: sys.gettickeraccumulation() <>`__ - - Having syntactic support for variable-sized unpacking came up again Contributing threads: - `syntactic shortcut - unpack to variably sized list <>`__ =============== Skipped Threads =============== - Deprecate PyRange_New() - interlocking dependencies on the path to a release see the first email to see what steps are needed for a release - Overflow in socketmodule.c? - Int literals and method calls To get access to methods on an int literal put a space after the number; ``1 .__class__`` - os.access versus os.exist - syntactic shortcut - unpack to variably sized list don't expect syntactic support variable-length sequences ain't happenin' From bac at OCF.Berkeley.EDU Sat Dec 18 08:24:44 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Dec 18 08:24:55 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules posixmodule.c, 2.300.8.10, 2.300.8.11 In-Reply-To: <20041218091927.E1961@bullseye.apana.org.au> References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer> <20041218091927.E1961@bullseye.apana.org.au> Message-ID: <41C3DB3C.4030707@ocf.berkeley.edu> Andrew MacIntyre wrote: > I don't see any possible way for those checkins to affect any platform > other than OS/2. > > 2 of the files are platform specific files (PC/os2emx/getpath.c, > PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is > contained within a platform specific #if/#endif: > Perhaps, but if you look at line 3299 you will notice that a comment is started but not closed off before the next comment on line 3304. How is that comment supposed to be closed off? -Brett From tim.peters at gmail.com Sat Dec 18 09:11:48 2004 From: tim.peters at gmail.com (Tim Peters) Date: Sat Dec 18 09:11:51 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules posixmodule.c, 2.300.8.10, 2.300.8.11 In-Reply-To: <41C3DB3C.4030707@ocf.berkeley.edu> References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer> <20041218091927.E1961@bullseye.apana.org.au> <41C3DB3C.4030707@ocf.berkeley.edu> Message-ID: <1f7befae0412180011484b44d8@mail.gmail.com> [Raymond, says test_glob and test_urllib fail on WinME now] [Andrew MacIntyre] >> I don't see any possible way for those checkins to affect any platform >> other than OS/2. >> >> 2 of the files are platform specific files (PC/os2emx/getpath.c, >> PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is >> contained within a platform specific #if/#endif: [Brett] > Perhaps, but if you look at line 3299 you will notice that a comment is started > but not closed off before the next comment on line 3304. How is that comment > supposed to be closed off? That's obviously a bug, and was introduced by the patch. Andrew wanted /* setup the pipe */ not /* setup the pipe But we don't get there unless PYOS_OS2 and PYCC_VACPP are defined, so that can't explain Raymond's problem. FWIW, the tests at issue pass on WinXP for me today w/ current CVS. From pedronis at bluewin.ch Sat Dec 18 16:03:54 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Sat Dec 18 16:02:53 2004 Subject: [Python-Dev] 2.4 news reaches interesting places In-Reply-To: <ca471dc204120814186f5dc423@mail.gmail.com> References: <ca471dc204120814186f5dc423@mail.gmail.com> Message-ID: <41C446DA.5080905@bluewin.ch> Guido van Rossum wrote: > I was pleasantly surprised to find a pointer to this article in a news > digest that the ACM emails me regularly (ACM TechNews). > > http://gcn.com/vol1_no1/daily-updates/28026-1.html > > One thing that bugs me: the article says 3 or 4 times that Python is > slow, each time with a refutation ("but it's so flexible", "but it's > fast enough") but still, they sure seem to harp on the point. This is > a PR issue that Python needs to fight -- any ideas? > I think that right now there is a promising ongoing happening that may be useful for this PR problem and Python perception in general, which is the Chandler project, because it's a piece of software that may end up in front of lot of people. It is both an opportunity and a risk. A not great success of Chandler especially if imputed to Python shortcomings would be bad. I think that people with long-time Python experience that feel like to contribute should try. A successful, responsive Chandler, I think, would be a good thing for Python. From andymac at bullseye.apana.org.au Sat Dec 18 10:33:31 2004 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Sat Dec 18 19:20:15 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modules posixmodule.c, 2.300.8.10, 2.300.8.11 In-Reply-To: <1f7befae0412180011484b44d8@mail.gmail.com> References: <000401c4e436$8eabb0c0$e841fea9@oemcomputer> <20041218091927.E1961@bullseye.apana.org.au> <41C3DB3C.4030707@ocf.berkeley.edu> <1f7befae0412180011484b44d8@mail.gmail.com> Message-ID: <20041218201510.N3850@bullseye.apana.org.au> On Sat, 18 Dec 2004, Tim Peters wrote: > [Raymond, says test_glob and test_urllib fail on WinME now] > > [Andrew MacIntyre] > >> I don't see any possible way for those checkins to affect any platform > >> other than OS/2. > >> > >> 2 of the files are platform specific files (PC/os2emx/getpath.c, > >> PC/os2vacpp/getpath.c), and the checkin to Modules/posixmodule.c is > >> contained within a platform specific #if/#endif: > > [Brett] > > Perhaps, but if you look at line 3299 you will notice that a comment is started > > but not closed off before the next comment on line 3304. How is that comment > > supposed to be closed off? > > That's obviously a bug, and was introduced by the patch. Andrew wanted > > /* setup the pipe */ > > not > > /* setup the pipe Fair cop guv. Long exposure to a language with comments of the latter form has been a PITA at times. :-( > But we don't get there unless PYOS_OS2 and PYCC_VACPP are defined, so > that can't explain Raymond's problem. > > FWIW, the tests at issue pass on WinXP for me today w/ current CVS. Good, though Raymond's report appeared to be focussed on the 2.3 branch. ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From python at rcn.com Sat Dec 18 19:24:07 2004 From: python at rcn.com (Raymond Hettinger) Date: Sat Dec 18 19:27:03 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Modulesposixmodule.c, 2.300.8.10, 2.300.8.11 In-Reply-To: <20041218201510.N3850@bullseye.apana.org.au> Message-ID: <000001c4e52e$c3bfcdc0$7d35c797@oemcomputer> > > FWIW, the tests at issue pass on WinXP for me today w/ current CVS. Tests pass here too. Raymond From stewart.midwinter at gmail.com Fri Dec 17 15:20:53 2004 From: stewart.midwinter at gmail.com (Stewart Midwinter) Date: Sat Dec 18 19:53:54 2004 Subject: [Python-Dev] re: 2.4 news reaches interesting places In-Reply-To: <864d37090412170320561ecb8@mail.gmail.com> References: <d4c0d4800412160513d16f13e@mail.gmail.com> <864d37090412170320561ecb8@mail.gmail.com> Message-ID: <d4c0d480041217062074703419@mail.gmail.com> On Fri, 17 Dec 2004 09:20:18 -0200, Carlos Ribeiro <carribeiro@gmail.com> wrote: > One possible marketing strategy is to use the adjective "fast" in a > broader sense. The Python slogan could be something like: "Programming > has never been any faster" -- this changes the playing ground, from > raw performance to *programming* performance. And sure, nothing beats > Python (the overall package) in this respect. It can deliver fast code > in a short time. Othere languages are faster to run, but take longer > to code... Parabens, Carlos, boa ideia! Carlos has said it well, Python has the edge in overall performance. Evidence seems to indicate a 3-4 times edge over development in Java, and this is a powerful argument. cheers, -- Stewart Midwinter stewart@midwinter.ca stewart.midwinter@gmail.com From p.f.moore at gmail.com Sat Dec 18 21:27:49 2004 From: p.f.moore at gmail.com (Paul Moore) Date: Sat Dec 18 21:27:52 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <41C0B32C.7040701@v.loewis.de> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> <41BE150F.9050100@v.loewis.de> <79990c6b04121402011d8965bd@mail.gmail.com> <41BF6DC9.8030500@v.loewis.de> <79990c6b041215014944f31097@mail.gmail.com> <41C0B32C.7040701@v.loewis.de> Message-ID: <79990c6b0412181227f5d4959@mail.gmail.com> On Wed, 15 Dec 2004 22:57:00 +0100, Martin v. L?wis <martin@v.loewis.de> wrote: > Paul Moore wrote: > > For a starter, what steps do you actually take to build a release? I > > assume that the first step is to build Python, by clicking on "build" > > in VS.NET. > > Yes. You can skip this step by just putting all the .pyds, dlls, and > .exes into the PCbuild directory. The packaging will try to pick them > up from there (and proceed if some are missing, like Tcl likely will). > > > Once you have that, is there a single script you run? > > Yes. Invoke Tools\msi\msi.py, using a Python that has pythonwin (i.e. > COM interopability). The only tricky part is that you need cabarc.exe, > which is included in VC, and in the platform SDK. OK, I've got a copy of the Python sources, and had a look. The change needed to msi.py to include libpythonXX.a in the installer looks simple. But I'm less sure as to where to build the file. It seems to me that msi.py is not the right place - that's focused on building the installer, not building the files to be installed. On the other hand, including it in the build process is a nuisance, as well, as it would add a dependency on mingw (or cygwin) to the MSVC build process. My feeling is that building libpythonXX.a should be a separate step, handled by a dedicated script, which gets run before msi.py. What do others (particularly Martin) think? Should I keep the steps separate, and focused on one task each, or should I incorporate the build of libpythonXX.a into msi.py, so that the installer build still requires just one step? Paul. From jcarlson at uci.edu Sat Dec 18 22:17:49 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat Dec 18 22:37:16 2004 Subject: [Python-Dev] mmap feature or bug? Message-ID: <20041218130908.9236.JCARLSON@uci.edu> Quick questions: Is mmap's inability to be subclassed a feature or bug? Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and others) a feature or bug? I would imagine they are bugs, but I just wanted to make sure and post the report/request in the proper location. Thank you, - Josiah From martin at v.loewis.de Sat Dec 18 23:34:11 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 18 23:34:12 2004 Subject: [Python-Dev] MinGW And The other Py2.4 issue In-Reply-To: <79990c6b0412181227f5d4959@mail.gmail.com> References: <BAY23-F609728FDC013615101711ABAA0@phx.gbl> <79990c6b041213015861230c01@mail.gmail.com> <41BE150F.9050100@v.loewis.de> <79990c6b04121402011d8965bd@mail.gmail.com> <41BF6DC9.8030500@v.loewis.de> <79990c6b041215014944f31097@mail.gmail.com> <41C0B32C.7040701@v.loewis.de> <79990c6b0412181227f5d4959@mail.gmail.com> Message-ID: <41C4B063.3020101@v.loewis.de> Paul Moore wrote: > OK, I've got a copy of the Python sources, and had a look. The change > needed to msi.py to include libpythonXX.a in the installer looks > simple. But I'm less sure as to where to build the file. It seems to > me that msi.py is not the right place - that's focused on building the > installer, not building the files to be installed. Don't worry about this: there is already quite some building going on in msi.py. If you look at the CVS copy of Tools/msi, you find that it now has a nmake makefile to build msisupport.dll, which replaces the VB scripts. It also extracts msvcr71.dll from the merge module (.msm) each time it is invoked. So having yet another build process would be just fine; adding it to the makefile would have the added advantage that nmake will compare time stamps for us (if it is easier to do in Python than in nmake, that would be a reason not to use nmake, though). > On the other hand, including it in the build process is a nuisance, as > well, as it would add a dependency on mingw (or cygwin) to the MSVC > build process. That is definitely undesirable. Lots of people build Python using the project files, and only few need the packaging to work. > My feeling is that building libpythonXX.a should be a separate step, > handled by a dedicated script, which gets run before msi.py. Making it separate is fine, as long as msi.py invokes/calls it. > What do others (particularly Martin) think? Should I keep the steps > separate, and focused on one task each, or should I incorporate the > build of libpythonXX.a into msi.py, so that the installer build still > requires just one step? Having the entire process involve as few manual steps as possible is definitely an important goal. Keeping it modular (in terms of splitting it into several files) is also a good idea, and one which does not conflict at all with the "fully automatic" goal. msi.py supports a config.py which allows to add customization. Putting cygwin_dir = r"C:\cygwin" into msi.py might be appropriate, with an option to set cygwin_dir to None, to indicate that cygwin should not be used in the build process. (similar to the way have_tcl already works). Regards, Martin From martin at v.loewis.de Sat Dec 18 23:47:29 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Dec 18 23:47:31 2004 Subject: [Python-Dev] mmap feature or bug? In-Reply-To: <20041218130908.9236.JCARLSON@uci.edu> References: <20041218130908.9236.JCARLSON@uci.edu> Message-ID: <41C4B381.7000808@v.loewis.de> Josiah Carlson wrote: > Is mmap's inability to be subclassed a feature or bug? No. It's a missing feature: it's not a bug, because nobody says this should work, and anybody trying will find out that it doesn't work, so nobody is tricked into believing it should work. The mmap type is not even documented; mmap.mmap is a function. It's not a feature, because (atleast IMO) there would be nothing wrong with making mmap.mmap a subclassable type. > Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and > others) a feature or bug? That is a bug. Slice assignments are supported, and so should __setslice__. Regards, Martin From mwh at python.net Sun Dec 19 17:34:55 2004 From: mwh at python.net (Michael Hudson) Date: Sun Dec 19 17:34:56 2004 Subject: [Python-Dev] mmap feature or bug? In-Reply-To: <20041218130908.9236.JCARLSON@uci.edu> (Josiah Carlson's message of "Sat, 18 Dec 2004 13:17:49 -0800") References: <20041218130908.9236.JCARLSON@uci.edu> Message-ID: <2mis6yl00g.fsf@starship.python.net> Josiah Carlson <jcarlson@uci.edu> writes: > Quick questions: > Is mmap's inability to be subclassed a feature or bug? Like Martin said, a missing feature. I'd also quite like the mmap module to have a C API, but as my use case is especially evil ( http://starship.python.net/crew/mwh/hacks/PPY.html ) I'm not going to push that hard :) > Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and > others) a feature or bug? I'm not entirely sure about this, though. Does mmapinstance.__setitem__(slice(1,2), 'a') work? AIUI __setslice__ and __getslice__ are out of favour these days (not sure though). Any patch to fix one of these would probably also fix the other almost as a sside-effect... Cheers, mwh -- ... with these conditions cam the realisation that ... nothing turned a perfectly normal healthy individual into a great political or military leader better than irreversible brain damage. -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From martin at v.loewis.de Sun Dec 19 19:13:55 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 19 19:13:56 2004 Subject: [Python-Dev] Python in education In-Reply-To: <41C02165.2010000@ocf.berkeley.edu> References: <41C02165.2010000@ocf.berkeley.edu> Message-ID: <41C5C4E3.7090001@v.loewis.de> Randy Chung wrote: > I'm going to be leading a class on Python at the University of > California, Berkeley next semester (starting in January). I'm > interested in using actual bugs in Python as exercises for the class, > the goal being 1) to give the students something "real" to work with, > and 2) to (hopefully) contribute back anything we manage to put together > to the Python dev team. Welcome to the club! I'm just running a class on development processes in open source software (i.e. with a somewhat different scope), where students already have fixed bugs in Mozilla and PHP - unfortunately, none of them were interested in Python. Nevertheless, this is quite fun for both the students, and myself - especially when the Mozilla guys do a review after 3h, the super review after 36h, and explain that they cannot commit the fix because the code is frozen (and then do after two weeks, well before the presentation in the class). > What I'm looking for are some oustanding bugs which the more experienced > devs feel would be straightforward to implement I'd look in the one-to-two-year old range of bugs. One risk in such a course is that some of the developers fixes the bug while your student is working on it, which is discouraging. How many bugs do you need? The *really* easy ones are documentation bugs, but I expect that students don't see this as a sufficient challenge. There are currently roughly 100 documentation bugs open. You can further filter out all assigned bugs; the unassigned ones are likely resting because nobody cares (yet). Looking at unassigned bugs starting from offset 300, I think all of the following bugs might be suitable: - 840065 - 839151 - 837242 (really trivial) - 828743 - 813986 > If you feel this is off topic, please feel free to reply to me off-list. This is clearly python-dev relevant. Regards, Martin From jcarlson at uci.edu Sun Dec 19 19:19:32 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun Dec 19 19:28:01 2004 Subject: [Python-Dev] mmap feature or bug? In-Reply-To: <2mis6yl00g.fsf@starship.python.net> References: <20041218130908.9236.JCARLSON@uci.edu> <2mis6yl00g.fsf@starship.python.net> Message-ID: <20041219101536.923B.JCARLSON@uci.edu> Michael Hudson <mwh@python.net> wrote: > > Josiah Carlson <jcarlson@uci.edu> writes: > > Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and > > others) a feature or bug? > > I'm not entirely sure about this, though. Does > > mmapinstance.__setitem__(slice(1,2), 'a') > > work? AIUI __setslice__ and __getslice__ are out of favour these days > (not sure though). > > Any patch to fix one of these would probably also fix the other almost > as a sside-effect... Accessing mmapi.__setslice__, mmapi.__getslice__, mmapi.__getitem__ and mmapi.__setitem__ fail with attribute errors. - Josiah From martin at v.loewis.de Sun Dec 19 23:16:57 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Dec 19 23:16:58 2004 Subject: [Python-Dev] MS VC compiler versions In-Reply-To: <41B08186.5070600@egenix.com> References: <41ADDED2.30707@egenix.com> <41AE1367.5000805@activestate.com> <41B08186.5070600@egenix.com> Message-ID: <41C5FDD9.1030907@v.loewis.de> M.-A. Lemburg wrote: > Here's an update of the list I currently have: I have * SDK Itanium compiler Microsoft (R) C/C++ Optimizing Compiler Version 13.10.2240.8 for IA-64 Copyright (C) Microsoft Corporation 1984-2002. All rights reserved. Regards, Martin From martin at v.loewis.de Mon Dec 20 08:47:37 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Dec 20 08:47:37 2004 Subject: [Python-Dev] MS VC compiler versions In-Reply-To: <41C68119.8070405@egenix.com> References: <41ADDED2.30707@egenix.com> <41AE1367.5000805@activestate.com> <41B08186.5070600@egenix.com> <41C5FDD9.1030907@v.loewis.de> <41C68119.8070405@egenix.com> Message-ID: <41C68399.1030704@v.loewis.de> M.-A. Lemburg wrote: > Thanks, Martin. Could you also give me the official version > number of the compiler or SDK ? It is "Windows Server 2003 Platform SDK, February 2003". Regards, Martin From arigo at tunes.org Mon Dec 20 13:31:32 2004 From: arigo at tunes.org (Armin Rigo) Date: Mon Dec 20 13:41:14 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python marshal.c, 1.79, 1.80 In-Reply-To: <E1CgMbv-00009V-PD@sc8-pr-cvs1.sourceforge.net> References: <E1CgMbv-00009V-PD@sc8-pr-cvs1.sourceforge.net> Message-ID: <20041220123132.GA14073@vicky.ecs.soton.ac.uk> Hi! A single-character diff... On Mon, Dec 20, 2004 at 04:25:59AM -0800, arigo@users.sourceforge.net wrote: > --- marshal.c 27 Jun 2004 16:51:46 -0000 1.79 > +++ marshal.c 20 Dec 2004 12:25:57 -0000 1.80 > @@ -893,7 +893,7 @@ > { > PyObject *x; > int version = Py_MARSHAL_VERSION; > - if (!PyArg_ParseTuple(args, "O|i:dumps", &x, version)) > + if (!PyArg_ParseTuple(args, "O|i:dumps", &x, &version)) > return NULL; > return PyMarshal_WriteObjectToString(x, version); > } Crash. Which means that there is no way in 2.4.0 to marshal an object in the old version format as a string -- you'd have to work around by writing a real file and reading it back :-( Armin From aahz at pythoncraft.com Mon Dec 20 18:17:53 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Dec 20 18:18:30 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python marshal.c, 1.79, 1.80 In-Reply-To: <20041220123132.GA14073@vicky.ecs.soton.ac.uk> References: <E1CgMbv-00009V-PD@sc8-pr-cvs1.sourceforge.net> <20041220123132.GA14073@vicky.ecs.soton.ac.uk> Message-ID: <20041220171753.GA26855@panix.com> On Mon, Dec 20, 2004, Armin Rigo wrote: > > Hi! > > A single-character diff... > > On Mon, Dec 20, 2004 at 04:25:59AM -0800, arigo@users.sourceforge.net wrote: >> --- marshal.c 27 Jun 2004 16:51:46 -0000 1.79 >> +++ marshal.c 20 Dec 2004 12:25:57 -0000 1.80 >> @@ -893,7 +893,7 @@ >> { >> PyObject *x; >> int version = Py_MARSHAL_VERSION; >> - if (!PyArg_ParseTuple(args, "O|i:dumps", &x, version)) >> + if (!PyArg_ParseTuple(args, "O|i:dumps", &x, &version)) >> return NULL; >> return PyMarshal_WriteObjectToString(x, version); >> } > > Crash. Which means that there is no way in 2.4.0 to marshal an object in the > old version format as a string -- you'd have to work around by writing a real > file and reading it back :-( Brown bag time? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From raymond.hettinger at verizon.net Mon Dec 20 18:48:53 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon Dec 20 18:52:42 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 Message-ID: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> [Armin] > Crash.? Which means that there is no way in 2.4.0 to marshal an object in the > > old version format as a string -- you'd have to work around by writing a real > > file and reading it back :-( [Aahz] > Brown bag time? Perhaps a rather quick Py2.4.1 would be in order. ? Ideally, it should include other?important fixes: ? *?Tim's critical thread bug ?? http://www.python.org/sf/1069160? ? *?Compile of _socket fails on 2.4 (SGI Orion only) ?? http://www.python.org/sf/1086642 ?? (waiting for a developer who has this set-up) ? *?sys.stdin segfaults on invalid stdin ?? http://www.python.org/sf/1084766 ?? (Jeff Epler has already isolated the problem) ? * truncated gzip file triggers zlibmodule segfault ?? http://www.python.org/sf/1083110 ?? (assigned to Andrew) ? * various installer issues (all currently assigned to Martin) ? * bad arg type to isspace in struct module ?? http://www.python.org/sf/1072182 ?? (assigned to me) ? * fix bug in StringIO.truncate - length not changed ?? http://www.python.org/sf/951915 ?? (assigned to me) ? * Fix for off-by-one bug in urllib.URLopener.retrieve ?? http://www.python.org/sf/810023 ?? (assigned to me) ? ? Raymond From barry at python.org Mon Dec 20 21:03:13 2004 From: barry at python.org (Barry Warsaw) Date: Mon Dec 20 21:03:17 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> Message-ID: <1103572993.6572.145.camel@geddy.wooz.org> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote: > Perhaps a rather quick Py2.4.1 would be in order. +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20041220/38908ce2/attachment.pgp From fredrik at pythonware.com Mon Dec 20 21:07:06 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon Dec 20 21:07:29 2004 Subject: [Python-Dev] Re: Re: [Python-checkins]python/dist/src/Pythonmarshal.c, 1.79, 1.80 References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> Message-ID: <cq7bd8$bmh$1@sea.gmane.org> Raymond Hettinger wrote: > Perhaps a rather quick Py2.4.1 would be in order. sounds like a good idea. +1 from here. </F> From jhylton at gmail.com Mon Dec 20 21:55:26 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Mon Dec 20 21:55:29 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <1103572993.6572.145.camel@geddy.wooz.org> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> Message-ID: <e8bf7a530412201255781556b7@mail.gmail.com> On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <barry@python.org> wrote: > On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote: > > > Perhaps a rather quick Py2.4.1 would be in order. > > +1 Nothing wrong with an incremental release, but none of these sound like critical bugs to me. Jeremy From irmen at xs4all.nl Mon Dec 20 22:32:57 2004 From: irmen at xs4all.nl (Irmen de Jong) Date: Mon Dec 20 22:33:00 2004 Subject: [Python-Dev] Latex problem when trying to build documentation Message-ID: <41C74509.4000109@xs4all.nl> I wanted to create a doc patch belonging to my patch #1062060, but got this error when trying to build the documentation from CVS: .... (/opt/PythonBugDay/python/dist/src/Doc/commontex/reportingbugs.tex Underfull \hbox (badness 10000) in paragraph at lines 13--17 []\OT1/ptm/m/n/10 All bug re-ports should be sub-mit-ted via the Python Bug Tra cker on Source-Forge ) [783] [784] Appendix C. (/opt/PythonBugDay/python/dist/src/Doc/commontex/license.tex [785] [786] [787] Underfull \hbox (badness 10000) in paragraph at lines 294--297 \OT1/ptm/m/n/10 The []\OT1/pcr/m/n/10 random \OT1/ptm/m/n/10 mod-ule in-cludes code based on a down-load from Overfull \hbox (4.8973pt too wide) in paragraph at lines 339--339 []\OT1/pcr/m/n/9 A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE CO PYRIGHT OWNER OR[] [788] Overfull \hbox (4.8973pt too wide) in paragraph at lines 376--376 []\OT1/pcr/m/n/9 FOR GAI_ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL[] Overfull \hbox (4.8973pt too wide) in paragraph at lines 376--376 []\OT1/pcr/m/n/9 HOWEVER CAUSED AND ON GAI_ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT[] [789] [790] [791] [792] [793] [794]) Underfull \hbox (badness 10000) in paragraph at lines 616--363 (./modlib.ind [795] [796] [797]) (./lib.ind [798] [799]) Package longtable Warning: Table widths have changed. Rerun LaTeX. ! LaTeX Error: \begin{description} on input line 59 ended by \end{document}. See the LaTeX manual or LaTeX Companion for explanation. Type H <return> for immediate help. ... l.380 \end{document} ? ! Emergency stop. ... l.380 \end{document} Output written on lib.dvi (805 pages, 2885748 bytes). Transcript written on lib.log. *** Session transcript and error messages are in /opt/PythonBugDay/python/dist/src/Doc/html/lib/lib.how. *** Exited with status 1. make: *** [html/lib/lib.html] Error 1 I dont really understand that latex stuff (used to, but that was a long time ago). What's the problem? I'm running this on Mandrake Linux 10.0 with Tetex 2.0.2. Regards, Irmen de Jong. From irmen at xs4all.nl Mon Dec 20 22:44:22 2004 From: irmen at xs4all.nl (Irmen de Jong) Date: Mon Dec 20 22:44:24 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> Message-ID: <41C747B6.6040500@xs4all.nl> Raymond Hettinger wrote: > Perhaps a rather quick Py2.4.1 would be in order. > > Ideally, it should include other important fixes: [...] > * Fix for off-by-one bug in urllib.URLopener.retrieve > http://www.python.org/sf/810023 > (assigned to me) Is http://www.python.org/sf/1062060 perhaps of similar importance? (fix for urllib.urlretrieve silently truncating download) --Irmen From python at rcn.com Mon Dec 20 23:25:34 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Dec 20 23:28:33 2004 Subject: [Python-Dev] Re:[Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <41C747B6.6040500@xs4all.nl> Message-ID: <002501c4e6e2$d30787e0$e841fea9@oemcomputer> > Raymond Hettinger wrote: > > Perhaps a rather quick Py2.4.1 would be in order. > > > > Ideally, it should include other important fixes: > [...] > > * Fix for off-by-one bug in urllib.URLopener.retrieve > > http://www.python.org/sf/810023 > > (assigned to me) > > Is http://www.python.org/sf/1062060 perhaps of similar importance? > (fix for urllib.urlretrieve silently truncating download) That seems reasonable to me. There is no point in having the error pass silently. Raymond From irmen at xs4all.nl Mon Dec 20 23:44:31 2004 From: irmen at xs4all.nl (Irmen de Jong) Date: Mon Dec 20 23:44:33 2004 Subject: [Python-Dev] Re:[Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <002501c4e6e2$d30787e0$e841fea9@oemcomputer> References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer> Message-ID: <41C755CF.2030305@xs4all.nl> Raymond Hettinger wrote: >>>* Fix for off-by-one bug in urllib.URLopener.retrieve >>> http://www.python.org/sf/810023 >>> (assigned to me) >> >>Is http://www.python.org/sf/1062060 perhaps of similar importance? >>(fix for urllib.urlretrieve silently truncating download) > > > That seems reasonable to me. There is no point in having the error > pass silently. Well, I wanted to make the patches that Johannes suggested, but ran into trouble when building the Python docs from CVS source (see my other message about this). Also, I'm not sure how a test-case should be constructed for this patch? Can the Python regression test download stuff as part of a test? Or is there some other way to make a testcase for this. --Irmen From tcdelaney at optusnet.com.au Tue Dec 21 00:00:09 2004 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Tue Dec 21 00:00:24 2004 Subject: [Python-Dev] Re:[Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer> <41C755CF.2030305@xs4all.nl> Message-ID: <005b01c4e6e7$a819bf30$4c00a8c0@ryoko> Irmen de Jong wrote: > Also, I'm not sure how a test-case should be constructed > for this patch? Can the Python regression test download stuff > as part of a test? Or is there some other way to make a > testcase for this. Hmm - perhaps start a server on the local machine at the start of the test, and tear it down at the end? you've then got full control of that server and can make it do whatever you want. Or replace the socket objects with mock objects? Tim Delaney From aahz at pythoncraft.com Tue Dec 21 00:25:39 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Dec 21 00:25:41 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <e8bf7a530412201255781556b7@mail.gmail.com> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> Message-ID: <20041220232539.GA10918@panix.com> On Mon, Dec 20, 2004, Jeremy Hylton wrote: > On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <barry@python.org> wrote: >> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote: >>> >>> Perhaps a rather quick Py2.4.1 would be in order. >> >> +1 > > Nothing wrong with an incremental release, but none of these sound > like critical bugs to me. You don't think a blowup in marshal is critical? Mind expanding on that? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis From jhylton at gmail.com Tue Dec 21 01:05:05 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Tue Dec 21 01:05:09 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <20041220232539.GA10918@panix.com> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> <20041220232539.GA10918@panix.com> Message-ID: <e8bf7a53041220160573d0e99e@mail.gmail.com> On Mon, 20 Dec 2004 18:25:39 -0500, Aahz <aahz@pythoncraft.com> wrote: > On Mon, Dec 20, 2004, Jeremy Hylton wrote: > > On Mon, 20 Dec 2004 15:03:13 -0500, Barry Warsaw <barry@python.org> wrote: > >> On Mon, 2004-12-20 at 12:48, Raymond Hettinger wrote: > >>> > >>> Perhaps a rather quick Py2.4.1 would be in order. > >> > >> +1 > > > > Nothing wrong with an incremental release, but none of these sound > > like critical bugs to me. > > You don't think a blowup in marshal is critical? Mind expanding on > that? An undocumented extension to marshal causes a segfault. It's certainly a bug worth fixing. It doesn't sound like a critical bug to me. Jeremy From tim.peters at gmail.com Tue Dec 21 01:26:25 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 21 01:26:29 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <e8bf7a53041220160573d0e99e@mail.gmail.com> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> <20041220232539.GA10918@panix.com> <e8bf7a53041220160573d0e99e@mail.gmail.com> Message-ID: <1f7befae04122016264c6a624d@mail.gmail.com> [Jeremy Hylton on a quick 2.4.1] >>> Nothing wrong with an incremental release, but none of these sound >>> like critical bugs to me. [Aahz] >> You don't think a blowup in marshal is critical? Mind expanding on >> that? [Jeremy] > An undocumented extension to marshal causes a segfault. It's > certainly a bug worth fixing. It doesn't sound like a critical bug to > me. The new optional ``version`` argument to marshal.dumps() is documented. The easiest way to see that is to look at 2.4's marshal.dumps() docs <wink>. Unfortunately, it was wholly untested. Still, it's a new-in-2.4 gimmick, and no pre-2.4 code could be using it. I suppose Armin found a use for it in 2.4, but I'm still scratching my head. If ZODB doesn't already depend on it, how useful can it be? QED WRT "my" critical thread bug, I asked that everyone pretend I hadn't submitted it until a month after 2.4 was released. That hasn't happened yet, so I refuse to admit it exists. FWIW, I'd press on with 2.3.5 first, while it can still attract some volunteer effort. From andrewm at object-craft.com.au Tue Dec 21 09:21:44 2004 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Tue Dec 21 09:21:39 2004 Subject: [Python-Dev] Py2.4 _sre uses uninitialised memory (Bug 1088891) Message-ID: <20041221082144.A8EE13C8E5@coffee.object-craft.com.au> _sre.c, data_stack_grow() in Py2.4 uses realloc()'ed memory without initialising the newly allocated memory. For complex regexps that require additional sre stack space, this ultimately results in a core dump or corrupted heap. Filling the newly allocated memory with 0x55 makes the problem more obvious (dies on a reference to 0x55555558) for me. See bug ID 1088891: http://sourceforge.net/tracker/index.php?func=detail&aid=1088891&group_id=5470&atid=105470 Can I be the only person who crafts diabolical regexps? Here, have a lend of my brown paper bag... -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From python at dynkin.com Tue Dec 21 13:13:12 2004 From: python at dynkin.com (George Yoshida) Date: Tue Dec 21 13:15:51 2004 Subject: [Python-Dev] Latex problem when trying to build documentation In-Reply-To: <41C74509.4000109@xs4all.nl> References: <41C74509.4000109@xs4all.nl> Message-ID: <41C81358.3080601@dynkin.com> Irmen de Jong wrote: > I wanted to create a doc patch belonging to my patch #1062060, > but got this error when trying to build the documentation from CVS: > [snip] > I dont really understand that latex stuff (used to, but that > was a long time ago). What's the problem? > Looks like the end tag of {description} has disappeared with this check-in: rhettinger@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Doc/lib > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23423 > > Modified Files: > libundoc.tex > Log Message: > SF bug #1084457: ossaudiodev no longer undocumented > > > > Index: libundoc.tex > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Doc/lib/libundoc.tex,v > retrieving revision 1.88 > retrieving revision 1.89 > diff -u -d -r1.88 -r1.89 > --- libundoc.tex 7 Aug 2004 19:21:59 -0000 1.88 > +++ libundoc.tex 14 Dec 2004 07:19:22 -0000 1.89 > @@ -71,11 +71,6 @@ > --- Convert "arbitrary" sound files to AIFF files; should probably > become a tool or demo. Requires the external program \program{sox}. > > -\item[\module{ossaudiodev}] > ---- Play audio data via the Open Sound System API. This is usable on > -Linux, some flavors of BSD, and some commercial \UNIX{} platforms. > -\end{description} > - > > \section{Obsolete \label{obsolete-modules}} George From niemeyer at conectiva.com Tue Dec 21 13:43:13 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Tue Dec 21 13:43:32 2004 Subject: [Python-Dev] Py2.4 _sre uses uninitialised memory (Bug 1088891) In-Reply-To: <20041221082144.A8EE13C8E5@coffee.object-craft.com.au> References: <20041221082144.A8EE13C8E5@coffee.object-craft.com.au> Message-ID: <20041221124313.GA7759@burma.localdomain> Hello Andrew, > _sre.c, data_stack_grow() in Py2.4 uses realloc()'ed memory without > initialising the newly allocated memory. For complex regexps that > require additional sre stack space, this ultimately results in a core > dump or corrupted heap. Filling the newly allocated memory with 0x55 > makes the problem more obvious (dies on a reference to 0x55555558) for > me. As I just reported in the bug, the problem is not initializing the allocated memory, but acknowledging memory reallocation in certain situations where it's reallocated outside of the main matching function. Have a look at the bug at http://python.org/sf/1072259 for more information and for a patch fixing the problem. Thanks for the report, -- Gustavo Niemeyer http://niemeyer.net From arigo at tunes.org Tue Dec 21 15:01:35 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Dec 21 15:11:28 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <1f7befae04122016264c6a624d@mail.gmail.com> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> <20041220232539.GA10918@panix.com> <e8bf7a53041220160573d0e99e@mail.gmail.com> <1f7befae04122016264c6a624d@mail.gmail.com> Message-ID: <20041221140135.GA18577@vicky.ecs.soton.ac.uk> Hi Tim, On Mon, Dec 20, 2004 at 07:26:25PM -0500, Tim Peters wrote: > Still, it's a new-in-2.4 gimmick, and no pre-2.4 code could be using > it. I suppose Armin found a use for it in 2.4, but I'm still > scratching my head. If ZODB doesn't already depend on it, how useful > can it be? QED Some code in the 'py' lib used to use marshal to send simple objects between the main process and a subprocess. We ran into trouble when we extended the idea to a subprocess that would actually run via ssh on a remote machine, and the remote machine's Python version didn't match the local one. The obvious quick fix was to set the 'version' argument to 0 and pretend that it would be fine forever. Now that we essentially can't use this trick any more because of the 2.4.0 bug, we reverted to repr/eval, which is quite slower (and actually not guaranteed to work across Python versions either: string escapes sometimes change). We avoid to use cPickle because we want to be sure that only "simple enough" objects are sent over this way -- essentially nothing that depends on a precise Python module to be installed and identical on both machines. Armin From pycon at python.org Tue Dec 21 15:57:18 2004 From: pycon at python.org (Steve Holden) Date: Tue Dec 21 15:57:19 2004 Subject: [Python-Dev] PyCon is coming - we need your help Message-ID: <20041221145718.34B591E4002@bag.python.org> Dear Python User: I wonder if you would be kind enough to take the time to read this email and help us to publicize PyCon DC 2005, being held March 23-26 at the Cafritz Conference Center of George Washington University. The Call for Participation went out some time ago, but it is a good time to remind people that the deadline for submissions is December 31. If you personally are thinking of submitting a paper then this can be a reminder to do so soon! We already have acceptances from two keynote speakers, and hope to announce them when a third is finalised shortly. As always you can find out about the conference at http://www.pycon.org/ http://www.python.org/pycon/ This year we are going to be able to accept credit card payments for the first time, which we hope will be more convenient for delegates. The registration fees this year are the same as for 2004: Early Bird (to Jan 22) $175 ($125 student) Regular (to Mar 19) $250 ($175 student) On-Site $300 ($225 student) A further announcment will be made when the registration site opens for business. In the meantime I would appreciate your assistance in posting this message via any channels you know of that have an interest in the Python language and its applications - publicity is the key to getting the most diverse group of people at PyCon. regards Steve Holden Chairman, PyCON DC 2005 -- PyCon DC 2005: The third Python Community Conference http://www.pycon.org/ http://www.python.org/pycon/ The scoop on Python implementations and applications From fredrik at pythonware.com Tue Dec 21 16:21:28 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue Dec 21 16:21:41 2004 Subject: [Python-Dev] Re: [Python-checkins]python/dist/src/Pythonmarshal.c, 1.79, 1.80 References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer><1103572993.6572.145.camel@geddy.wooz.org><e8bf7a530412201255781556b7@mail.gmail.com><20041220232539.GA10918@panix.com><e8bf7a53041220160573d0e99e@mail.gmail.com><1f7befae04122016264c6a624d@mail.gmail.com> <20041221140135.GA18577@vicky.ecs.soton.ac.uk> Message-ID: <cq9f1l$2rs$1@sea.gmane.org> Armin Rigo wrote: > Some code in the 'py' lib used to use marshal to send simple objects between > the main process and a subprocess. We ran into trouble when we extended the > idea to a subprocess that would actually run via ssh on a remote machine, and > the remote machine's Python version didn't match the local one. The obvious > quick fix was to set the 'version' argument to 0 and pretend that it would be > fine forever. this is a rather common use case. </F> From irmen at xs4all.nl Tue Dec 21 19:57:43 2004 From: irmen at xs4all.nl (Irmen de Jong) Date: Tue Dec 21 19:57:44 2004 Subject: [Python-Dev] Latex problem when trying to build documentation In-Reply-To: <41C81358.3080601@dynkin.com> References: <41C74509.4000109@xs4all.nl> <41C81358.3080601@dynkin.com> Message-ID: <41C87227.1070107@xs4all.nl> George Yoshida wrote: > Irmen de Jong wrote: > > > I wanted to create a doc patch belonging to my patch #1062060, > > but got this error when trying to build the documentation from CVS: > > [snip] > > I dont really understand that latex stuff (used to, but that > > was a long time ago). What's the problem? > > > > Looks like the end tag of {description} has disappeared with this > check-in: [...] Out of curiousity, how did you find this? --Irmen From irmen at xs4all.nl Tue Dec 21 20:00:22 2004 From: irmen at xs4all.nl (Irmen de Jong) Date: Tue Dec 21 20:00:25 2004 Subject: [Python-Dev] Re:[Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <005b01c4e6e7$a819bf30$4c00a8c0@ryoko> References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer> <41C755CF.2030305@xs4all.nl> <005b01c4e6e7$a819bf30$4c00a8c0@ryoko> Message-ID: <41C872C6.8040505@xs4all.nl> Tim Delaney wrote: > Irmen de Jong wrote: > >> Also, I'm not sure how a test-case should be constructed >> for this patch? Can the Python regression test download stuff >> as part of a test? Or is there some other way to make a >> testcase for this. > > > Hmm - perhaps start a server on the local machine at the start of the > test, and tear it down at the end? you've then got full control of that > server and can make it do whatever you want. > > Or replace the socket objects with mock objects? Thanks for those suggestions. Let's see what I can concoct. Never made (or studied in detail) a python regression test case before so now is a good time :) --Irmen From jlg at dds.nl Tue Dec 21 22:06:45 2004 From: jlg at dds.nl (Johannes Gijsbers) Date: Tue Dec 21 22:03:38 2004 Subject: [Python-Dev] Re:[Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <41C872C6.8040505@xs4all.nl> References: <002501c4e6e2$d30787e0$e841fea9@oemcomputer> <41C755CF.2030305@xs4all.nl> <005b01c4e6e7$a819bf30$4c00a8c0@ryoko> <41C872C6.8040505@xs4all.nl> Message-ID: <20041221210645.GB9105@authsmtp.dds.nl> On Tue, Dec 21, 2004 at 08:00:22PM +0100, Irmen de Jong wrote: > Thanks for those suggestions. > Let's see what I can concoct. > > Never made (or studied in detail) a python regression test case > before so now is a good time :) You might be especially interested in 'class urlopen_HttpTests' in test_urllib.py and all the Mock* classes in test_urllib2.py. BTW, don't hesitate to ask for help in the bug/patch. I'm happy to help, though I might be a bit slow to answer. Cheers, Johannes From jlg at dds.nl Tue Dec 21 22:20:15 2004 From: jlg at dds.nl (Johannes Gijsbers) Date: Tue Dec 21 22:17:08 2004 Subject: [Python-Dev] Latex problem when trying to build documentation In-Reply-To: <41C87227.1070107@xs4all.nl> References: <41C74509.4000109@xs4all.nl> <41C81358.3080601@dynkin.com> <41C87227.1070107@xs4all.nl> Message-ID: <20041221212015.GC9105@authsmtp.dds.nl> On Tue, Dec 21, 2004 at 07:57:43PM +0100, Irmen de Jong wrote: > >Looks like the end tag of {description} has disappeared with this > >check-in: > > [...] > > Out of curiousity, how did you find this? Once you filter through all the noise LaTeX generates there's actually a pretty clear error message: "\begin{description} on input line 59 ended by \end{document}." That's obviously not right. I've just fixed it, by the way. Cheers, Johannes From tim.peters at gmail.com Tue Dec 21 23:21:29 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 21 23:26:46 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <20041221140135.GA18577@vicky.ecs.soton.ac.uk> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> <20041220232539.GA10918@panix.com> <e8bf7a53041220160573d0e99e@mail.gmail.com> <1f7befae04122016264c6a624d@mail.gmail.com> <20041221140135.GA18577@vicky.ecs.soton.ac.uk> Message-ID: <1f7befae041221142115d3fd04@mail.gmail.com> [Armin Rigo] > Some code in the 'py' lib used to use marshal to send simple objects between > the main process and a subprocess. We ran into trouble when we extended the > idea to a subprocess that would actually run via ssh on a remote machine, and > the remote machine's Python version didn't match the local one. The obvious > quick fix was to set the 'version' argument to 0 and pretend that it would be > fine forever. Yes, I believed you had *some* use for it <wink>. > Now that we essentially can't use this trick any more because of the 2.4.0 > bug, Well, you can still use 2.3.4 -- or wait for 2.4.1 -- or use your patched 2.4. Or use the stock 2.4, but set up a "marshal server" running 2.3.4 <heh>. > we reverted to repr/eval, which is quite slower (and actually not guaranteed to > work across Python versions either: string escapes sometimes change). Really? The precise rules str's __repr__ uses for which escapes to produce certainly change, but I don't recall any case outside Unicodeland where a new string escape was ever introduced. So, e.g., current Python str.__repr__() produces '\n' for a newline, while long-ago Pythons produced '\012', but all versions of Python *accept* either form of escape. The biggest change of this kind was moving from octal escapes to hex escapes in Python 2.1, but hex escapes have always been accepted -- repr() just didn't produce them before 2.1. > We avoid to use cPickle because we want to be sure that only "simple > enough" objects are sent over this way -- essentially nothing that depends on > a precise Python module to be installed and identical on both machines. It's possible (but irksome) to subclass pickle.py's Pickler class, and override its save_global() and save_inst() methods. For example, replace them with one-liners that just raise an exception. Then any attempt to pickle an object requiring a specific module will raise that exception. But if you're worried about speed, going thru pickle.py is significantly slower than going thru repr(). From tim.peters at gmail.com Tue Dec 21 23:21:29 2004 From: tim.peters at gmail.com (Tim Peters) Date: Tue Dec 21 23:52:17 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <20041221140135.GA18577@vicky.ecs.soton.ac.uk> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> <20041220232539.GA10918@panix.com> <e8bf7a53041220160573d0e99e@mail.gmail.com> <1f7befae04122016264c6a624d@mail.gmail.com> <20041221140135.GA18577@vicky.ecs.soton.ac.uk> Message-ID: <1f7befae041221142115d3fd04@mail.gmail.com> [Armin Rigo] > Some code in the 'py' lib used to use marshal to send simple objects between > the main process and a subprocess. We ran into trouble when we extended the > idea to a subprocess that would actually run via ssh on a remote machine, and > the remote machine's Python version didn't match the local one. The obvious > quick fix was to set the 'version' argument to 0 and pretend that it would be > fine forever. Yes, I believed you had *some* use for it <wink>. > Now that we essentially can't use this trick any more because of the 2.4.0 > bug, Well, you can still use 2.3.4 -- or wait for 2.4.1 -- or use your patched 2.4. Or use the stock 2.4, but set up a "marshal server" running 2.3.4 <heh>. > we reverted to repr/eval, which is quite slower (and actually not guaranteed to > work across Python versions either: string escapes sometimes change). Really? The precise rules str's __repr__ uses for which escapes to produce certainly change, but I don't recall any case outside Unicodeland where a new string escape was ever introduced. So, e.g., current Python str.__repr__() produces '\n' for a newline, while long-ago Pythons produced '\012', but all versions of Python *accept* either form of escape. The biggest change of this kind was moving from octal escapes to hex escapes in Python 2.1, but hex escapes have always been accepted -- repr() just didn't produce them before 2.1. > We avoid to use cPickle because we want to be sure that only "simple > enough" objects are sent over this way -- essentially nothing that depends on > a precise Python module to be installed and identical on both machines. It's possible (but irksome) to subclass pickle.py's Pickler class, and override its save_global() and save_inst() methods. For example, replace them with one-liners that just raise an exception. Then any attempt to pickle an object requiring a specific module will raise that exception. But if you're worried about speed, going thru pickle.py is significantly slower than going thru repr(). From tim.peters at gmail.com Tue Dec 21 23:21:29 2004 From: tim.peters at gmail.com (Tim Peters) Date: Wed Dec 22 00:11:30 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <20041221140135.GA18577@vicky.ecs.soton.ac.uk> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> <20041220232539.GA10918@panix.com> <e8bf7a53041220160573d0e99e@mail.gmail.com> <1f7befae04122016264c6a624d@mail.gmail.com> <20041221140135.GA18577@vicky.ecs.soton.ac.uk> Message-ID: <1f7befae041221142115d3fd04@mail.gmail.com> [Armin Rigo] > Some code in the 'py' lib used to use marshal to send simple objects between > the main process and a subprocess. We ran into trouble when we extended the > idea to a subprocess that would actually run via ssh on a remote machine, and > the remote machine's Python version didn't match the local one. The obvious > quick fix was to set the 'version' argument to 0 and pretend that it would be > fine forever. Yes, I believed you had *some* use for it <wink>. > Now that we essentially can't use this trick any more because of the 2.4.0 > bug, Well, you can still use 2.3.4 -- or wait for 2.4.1 -- or use your patched 2.4. Or use the stock 2.4, but set up a "marshal server" running 2.3.4 <heh>. > we reverted to repr/eval, which is quite slower (and actually not guaranteed to > work across Python versions either: string escapes sometimes change). Really? The precise rules str's __repr__ uses for which escapes to produce certainly change, but I don't recall any case outside Unicodeland where a new string escape was ever introduced. So, e.g., current Python str.__repr__() produces '\n' for a newline, while long-ago Pythons produced '\012', but all versions of Python *accept* either form of escape. The biggest change of this kind was moving from octal escapes to hex escapes in Python 2.1, but hex escapes have always been accepted -- repr() just didn't produce them before 2.1. > We avoid to use cPickle because we want to be sure that only "simple > enough" objects are sent over this way -- essentially nothing that depends on > a precise Python module to be installed and identical on both machines. It's possible (but irksome) to subclass pickle.py's Pickler class, and override its save_global() and save_inst() methods. For example, replace them with one-liners that just raise an exception. Then any attempt to pickle an object requiring a specific module will raise that exception. But if you're worried about speed, going thru pickle.py is significantly slower than going thru repr(). From python at dynkin.com Wed Dec 22 00:22:42 2004 From: python at dynkin.com (George Yoshida) Date: Wed Dec 22 00:22:25 2004 Subject: [Python-Dev] Latex problem when trying to build documentation In-Reply-To: <41C87227.1070107@xs4all.nl> References: <41C74509.4000109@xs4all.nl> <41C81358.3080601@dynkin.com> <41C87227.1070107@xs4all.nl> Message-ID: <41C8B042.40102@dynkin.com> Irmen de Jong wrote: >> Looks like the end tag of {description} has disappeared with this >> check-in: > > > [...] > > Out of curiousity, how did you find this? > Well, I periodically download daily snapshots, so I compiled TeX files with them. The snapshot from Dec-11 was OK, but with the one from Dec-17 I couldn't compile. Once you're sure in what period the bug crept into the source, all you need to do is to check every commit applied to files under src/Doc directory in that period. Anyway, I don't know TeX well enough to narrow down the bug from the error log. It's too verbose for me. George From titus at caltech.edu Sun Dec 19 11:13:24 2004 From: titus at caltech.edu (Titus Brown) Date: Wed Dec 22 06:17:49 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. Message-ID: <20041219101324.GA17049@caltech.edu> Hello all, per previous discussion, http://mail.python.org/pipermail/python-dev/2004-October/049495.html I'd like to push a trivial little patch to sgmllib (#1087808) on you gents, in exchange for my reviews & effort etc. on 10 other patches. Without further ado: No-brainers: 1055159 -- a docstring+docs update to CGIHTTPServer describing already- existing behavior. Recommend apply. 1037974 -- fix HTTP digest auth for broken servers, e.g. LiveJournal. Trivial code fix, should break nothing. Recommend apply + backport. 1028908 -- JJ Lee's updates to urllib2. Passes regr tests, by an original author of much of the code (I think). Recommend apply. 901480 -- patch to urllib2.parse_http_list (bug 735248). Works. Updated patch. Recommend apply + backport. 827559 -- SimpleHTTPServer redirects to 'dir/' when asked for 'dir'. This behavior mimics common behavior online and fixes a problem with relative URLs when clicking on files within 'dir'. Recommend apply. 810023 -- fixes off-by-one bug in urllib reporthook. regr tests & all. Good stuff. Recommend apply + backport. 893642 -- adds allow_none option to SimpleXMLRPCServer & associated classes. Doesn't change default behavior. Recommend apply. 755670 -- modify HTMLParser to accept clearly broken HTML. Recommend reject. Slightly more complicated: 1067760 -- float-->long conversion on fileobj.seek calls, rather than float-->int. Permits larger floats (2.0**62) to match large int (2**62) arguments. rhettinger marked as "won't fix" in the original bug report; this seems like a clean solution, tho. Recommend apply. 755660 -- should HTMLParser fail on all bad input, or do best effort? I'd recommend more sweeping changes where must-fail situations are distinguished from fails-by-default situations. Alternatively take a stand and say "nein!" once and for all. (See my comment for more information.) -- For no particularly good reason, all of these were tested against the current CVS HEAD rather than 2.4. All of them should be trivial to backport, although I think only a few are real problems worthy of the effort. -- I'm kind of curious to see how this goes, I must admit ;). Please CC me on replies so I can listen in... One comment to Martin: it clearly isn't worth the effort of reviewing 10 patches to push a patch the size of my sgmllib patch. On the other hand, it's nice to have a guarantee & it's an educational experience, that's for sure. A 5:1 ratio might be more reasonable, since that in practice will mean 1 serious patch, 2 or 3 updates, and 1 drop-dead easy patch. cheers, --titus From arigo at tunes.org Wed Dec 22 13:53:37 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed Dec 22 14:03:26 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Pythonmarshal.c, 1.79, 1.80 In-Reply-To: <1f7befae041221142115d3fd04@mail.gmail.com> References: <000001c4e6bc$483e5060$e0ae958d@oemcomputer> <1103572993.6572.145.camel@geddy.wooz.org> <e8bf7a530412201255781556b7@mail.gmail.com> <20041220232539.GA10918@panix.com> <e8bf7a53041220160573d0e99e@mail.gmail.com> <1f7befae04122016264c6a624d@mail.gmail.com> <20041221140135.GA18577@vicky.ecs.soton.ac.uk> <1f7befae041221142115d3fd04@mail.gmail.com> Message-ID: <20041222125337.GA11426@vicky.ecs.soton.ac.uk> Hi Tim, On Tue, Dec 21, 2004 at 05:21:29PM -0500, Tim Peters wrote: > > we reverted to repr/eval, which is quite slower (and actually not guaranteed to > > work across Python versions either: string escapes sometimes change). > > Really? The precise rules str's __repr__ uses for which escapes to > produce certainly change, but I don't recall any case outside > Unicodeland where a new string escape was ever introduced. My mistake, I seemed to remember something along these lines but you are obviously right. Python 1.5.2 reads all escapes of a recent Python just fine. Armin From andre.duque at varig.com Wed Dec 22 16:34:23 2004 From: andre.duque at varig.com (=?ISO-8859-1?Q?Andr=E9_Amram_Duque?=) Date: Wed Dec 22 16:36:14 2004 Subject: [Python-Dev] Problem with ./configure (py2.3.4) Message-ID: <41C993FF.2030907@varig.com> I have problem with re-install python 2.3.4, when I execute ./configure is appear one message in config.log, follow below : configure:1710: gcc conftest.cc >&5 gcc: installation problem, cannot exec `cc1plus': No such file or directory configure:1713: $? = 1 My gnu/linux is 2.6.8-1-386(debian/sarge) Somebody could help me? Regards, -- Andre Amram Duque Consultor de Sistemas VARIG Brazilian Airlines - RIONS GGTI - Gerencia Geral de Tecnologia da Informacao Tel.: + 55-0XX-21-3814-5920 Fax:: + 55-0XX-21-3814-5796 e-mail: andre.duque@varig.com From fredrik at pythonware.com Wed Dec 22 17:04:04 2004 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed Dec 22 17:04:08 2004 Subject: [Python-Dev] Re: Problem with ./configure (py2.3.4) References: <41C993FF.2030907@varig.com> Message-ID: <cqc5ti$bov$1@sea.gmane.org> hi andre, >I have problem with re-install python 2.3.4, when I execute ./configure is appear one message in >config.log, follow below : > configure:1710: gcc conftest.cc >&5 > gcc: installation problem, cannot exec `cc1plus': No such file or directory > configure:1713: $? = 1 > > My gnu/linux is 2.6.8-1-386(debian/sarge) > > Somebody could help me? Since this is most likely a problem with your local configuration, it's pretty much off-topic for python-dev. I suggest asking for help on comp.lang.python instead. (but feel free to return if you can confirm that this really is a problem with the ./configure file, and that the problem is still there in Python 2.4). </F> From jhylton at gmail.com Wed Dec 22 23:34:20 2004 From: jhylton at gmail.com (Jeremy Hylton) Date: Wed Dec 22 23:34:24 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <20041219101324.GA17049@caltech.edu> References: <20041219101324.GA17049@caltech.edu> Message-ID: <e8bf7a5304122214347a9dd294@mail.gmail.com> On Sun, 19 Dec 2004 02:13:24 -0800, Titus Brown <titus@caltech.edu> wrote: > Hello all, > > per previous discussion, > > http://mail.python.org/pipermail/python-dev/2004-October/049495.html > > I'd like to push a trivial little patch to sgmllib (#1087808) on you > gents, in exchange for my reviews & effort etc. on 10 other patches. I got started on these this morning, will likely finish them tomorrow. It would be perverse to apply your patch last, wouldn't it? Jeremy From martin at v.loewis.de Wed Dec 22 23:40:26 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Dec 22 23:40:22 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <e8bf7a5304122214347a9dd294@mail.gmail.com> References: <20041219101324.GA17049@caltech.edu> <e8bf7a5304122214347a9dd294@mail.gmail.com> Message-ID: <41C9F7DA.1040408@v.loewis.de> Jeremy Hylton wrote: > I got started on these this morning, will likely finish them tomorrow. > It would be perverse to apply your patch last, wouldn't it? It turns out that Titus' patch might be more involved than he thought it would be. In any case, the review itself is a highly appreciated contribution. In the hope that this progress can trigger more contributions like this, I'll happily reduce the requirements to 5 reviews. Regards, Martin From bac at ocf.berkeley.edu Thu Dec 23 02:15:11 2004 From: bac at ocf.berkeley.edu (Brett C.) Date: Thu Dec 23 02:15:19 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <41C9F7DA.1040408@v.loewis.de> References: <20041219101324.GA17049@caltech.edu> <e8bf7a5304122214347a9dd294@mail.gmail.com> <41C9F7DA.1040408@v.loewis.de> Message-ID: <41CA1C1F.8080803@ocf.berkeley.edu> Martin v. L?wis wrote: > Jeremy Hylton wrote: > >> I got started on these this morning, will likely finish them tomorrow. >> It would be perverse to apply your patch last, wouldn't it? > > > It turns out that Titus' patch might be more involved than he thought > it would be. > > In any case, the review itself is a highly appreciated contribution. > > In the hope that this progress can trigger more contributions like this, > I'll happily reduce the requirements to 5 reviews. > And to help this along I am willing to officially toss my hat into the fray by being another person who will review a patch if someone does 5 reviews (sans anything that doesn't work on OS X, which leaves out Tkinter). -Brett From titus at caltech.edu Thu Dec 23 02:34:23 2004 From: titus at caltech.edu (Titus Brown) Date: Thu Dec 23 03:09:46 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <41C9F7DA.1040408@v.loewis.de> References: <20041219101324.GA17049@caltech.edu> <e8bf7a5304122214347a9dd294@mail.gmail.com> <41C9F7DA.1040408@v.loewis.de> Message-ID: <20041223013423.GA24543@caltech.edu> -> Jeremy Hylton wrote: -> >I got started on these this morning, will likely finish them tomorrow. -> > It would be perverse to apply your patch last, wouldn't it? -> -> It turns out that Titus' patch might be more involved than he thought -> it would be. *shrug* that's life ;). I stole my patch from the other HTMLParser & thought that would be sufficient; now I'll have to fix both! -> In any case, the review itself is a highly appreciated contribution. It was very educational; just wish I could remember to always submit context diffs! <sigh> The only patch that I think deserves some actual discussion - here or on c.l.p, not sure which -- is patch 755660, which deals with HTMLParser.HTMLParser. The goal of the original submitter was to allow subclasses of HTMLParser deal with bad HTML in a more robust way; essentially this comes down to allowing returns from self.error() calls. I have now come across the same problem in my work with PBP (pbp.berlios.de): it turns out that many Web pages (such as the SourceForge mailman admindb page...) contain errors that cause HTMLParser to raise an exception. It's simply not possible to reliably change this behavior within either htmllib.HTMLParser or HTMLParser.HTMLParser as they're currently written. This is a big problem for people basing packages on either HTMLParser class. An additional problem is that both HTMLParser.HTMLParser and htmllib.HTMLParser are based on other classes that call self.error(), so those base classes would have to altered to fit the new behavior. What I proposed doing in my comment on patch 755660 was changing HTMLParser.HTMLParser (and its base class markupbase, too) to call _fail() when a hard failure was called for, and otherwise to call error() and proceed parsing on an as-best-possible basis. This wouldn't change the *behavior* of the existing code, but would allow for it to be overridden when necessary. Right now the error() call is undocumented and so it's probably ok to change what happens upon return. As is it can leave the parser in a borked state upon return, and that's the behavior I propose to fix. I'd of course be willing to do the work & submit the more involved patch. Your opinions are not only welcome but (as I understand it) necessary ;). cheers, --titus From gvanrossum at gmail.com Thu Dec 23 03:19:52 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Dec 23 03:19:55 2004 Subject: [Python-Dev] Python for Series 60 released In-Reply-To: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com> References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com> Message-ID: <ca471dc2041222181945d20394@mail.gmail.com> Python runs on Nokia cell phones (the high-end ones, anyway) and has support from Nokia! Pretty cool all around. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ---------- Forwarded message ---------- From: Michele.Marchetti@nokia.com <Michele.Marchetti@nokia.com> Date: Wed, 22 Dec 2004 11:59:55 +0200 Subject: python for Series 60 released To: guido@python.org Hello, like promised during EuroPython I let you know about the fact that Python for Symbian Series 60 is released. You may find it in www.forum.nokia.com by choosing under "resources" "Tools and SDK's" main page. Have a nice Christmas time. Best Regards, Michele Marchetti From bob at redivi.com Thu Dec 23 03:59:21 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Dec 23 03:59:27 2004 Subject: [Python-Dev] Python for Series 60 released In-Reply-To: <ca471dc2041222181945d20394@mail.gmail.com> References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com> <ca471dc2041222181945d20394@mail.gmail.com> Message-ID: <A544CCC8-548E-11D9-B983-000A9567635C@redivi.com> Merry Christmas to me! I actually purchased a Nokia Series 60 phone when this was first announced, in hopes that this would be available soon. A littler later than I'd have liked, but better than never :) -bob On Dec 22, 2004, at 9:19 PM, Guido van Rossum wrote: > Python runs on Nokia cell phones (the high-end ones, anyway) and has > support from Nokia! > > Pretty cool all around. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > > ---------- Forwarded message ---------- > From: Michele.Marchetti@nokia.com <Michele.Marchetti@nokia.com> > Date: Wed, 22 Dec 2004 11:59:55 +0200 > Subject: python for Series 60 released > To: guido@python.org > > Hello, > > like promised during EuroPython I let you know about the fact that > Python for Symbian Series 60 is released. > You may find it in www.forum.nokia.com by choosing under "resources" > "Tools and SDK's" main page. > > Have a nice Christmas time. > Best Regards, > Michele Marchetti > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/bob%40redivi.com From kbk at shore.net Thu Dec 23 05:09:12 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Dec 23 05:09:42 2004 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200412230409.iBN49ClA021597@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 257 open ( -2) / 2715 closed ( +8) / 2972 total ( +6) Bugs : 807 open (-15) / 4717 closed (+32) / 5524 total (+17) RFE : 163 open ( +3) / 139 closed ( +0) / 302 total ( +3) New / Reopened Patches ______________________ sgmllib.SGMLParser does not unescape attribute values; patch (2004-12-18) http://python.org/sf/1087808 opened by Titus Brown Patch for bug 1088077 (2004-12-19) http://python.org/sf/1088078 opened by Mike Meyer msi.py patch to build mingw library (2004-12-20) CLOSED http://python.org/sf/1088716 opened by Paul Moore acknowledge signals in non-main threads (2004-12-21) http://python.org/sf/1088832 opened by Andrew Langmead Patches Closed ______________ CGIHTTPServer can't redirect (2004-10-27) http://python.org/sf/1055159 closed by jhylton urllib2 HTTP digest authentication fix (2004-09-30) http://python.org/sf/1037974 closed by jhylton fix bsddb memory leaks (2004-06-06) http://python.org/sf/967763 closed by greg fast dictionary lookup by name (2002-09-07) http://python.org/sf/606098 closed by rhettinger add time elapsed to gc debug output (2003-06-26) http://python.org/sf/760990 closed by rhettinger Argument passing from /usr/bin/idle2.3 to idle.py (2003-11-30) http://python.org/sf/851459 closed by jafo fix bug in StringIO.truncate - length not changed (2004-05-11) http://python.org/sf/951915 closed by rhettinger msi.py patch to build mingw library (2004-12-21) http://python.org/sf/1088716 closed by loewis New / Reopened Bugs ___________________ inspect.py module (2004-12-18) CLOSED http://python.org/sf/1087551 opened by sprasanna199 mmap instance method access bug (2004-12-18) http://python.org/sf/1087735 opened by Josiah Carlson Mac: make frameworkinstall skips docs, scripts (2004-12-19) http://python.org/sf/1087737 opened by Jack Jansen example code not working (2004-12-19) CLOSED http://python.org/sf/1087975 opened by perica CGIHTTPServer: directories/scripts with spaces in their name (2004-12-19) http://python.org/sf/1088039 opened by LT [PATCH] tty needs a way to restore the terminal mode. (2004-12-19) http://python.org/sf/1088077 opened by Mike Meyer Comments regarding 'macintosh' behaviour wrong for MacOS X (2004-12-20) CLOSED http://python.org/sf/1088119 opened by James Matthew Farrow zlib decompressobj documentation typo (2004-12-20) CLOSED http://python.org/sf/1088206 opened by Scott David Daniels calculation wrong rounding (2004-12-20) CLOSED http://python.org/sf/1088563 opened by Sebastian Rockel _sre.c references uninitialised memory (2004-12-21) CLOSED http://python.org/sf/1088891 opened by Andrew McNamara need siginterrupt() on Linux - impossible to do timeouts (2004-12-21) http://python.org/sf/1089358 opened by Jason segfault/assert in tokenizer (2004-12-21) http://python.org/sf/1089395 opened by Walter D?rwald Carbon.Res misses GetIndString (2004-12-21) http://python.org/sf/1089399 opened by Jack Jansen Carbon.File.FSCatalogInfo.createDate implementation (2004-12-22) http://python.org/sf/1089624 opened by Ronald Oussoren _DummyThread() objects not freed from threading._active map (2004-12-22) http://python.org/sf/1089632 opened by saravanand Carbon.File.FSCatalogInfo.createDate implementation (2004-12-22) CLOSED http://python.org/sf/1089643 opened by Ronald Oussoren special methods become static (2004-11-14) http://python.org/sf/1066490 reopened by kquick mmap missing offset parameter (2004-12-22) http://python.org/sf/1089974 opened by James Y Knight exec scoping problem (2004-12-22) http://python.org/sf/1089978 opened by Kevin Quick Defaults in ConfigParser.get overrides section values (2004-12-22) http://python.org/sf/1090076 opened by Gabriel Genellina presentation typo in lib: 6.21.4.2 How callbacks are called (2004-12-22) http://python.org/sf/1090139 opened by Jesse Weinstein Bugs Closed ___________ Tests fail instead of skip (2004-12-11) http://python.org/sf/1083645 closed by bcannon readline module doesn't build on MacOSX (2004-12-07) http://python.org/sf/1081045 closed by bcannon Python 2.4 crashes (2004-12-12) http://python.org/sf/1083793 closed by axel_kaiser inspect.py module (2004-12-18) http://python.org/sf/1087551 closed by jlgijsbers Possible error during LINKCC check in Configure.in (2004-06-25) http://python.org/sf/980127 closed by bcannon example code not working (2004-12-19) http://python.org/sf/1087975 closed by rhettinger Shared object modules in Windows have no __file__. (2003-10-05) http://python.org/sf/818315 closed by loewis Easier-to-create alternative Python installer for Windows (2003-08-22) http://python.org/sf/793078 closed by loewis datetime module documentation missing critical detail (2004-12-17) http://python.org/sf/1087216 closed by rhettinger "Limitations" section of profiler docs is incorrect (2004-11-15) http://python.org/sf/1066607 closed by rhettinger Incorrect error message (somewhat) (2004-12-04) http://python.org/sf/1079011 closed by rhettinger test_shelve failures (2004-12-01) http://python.org/sf/1076819 closed by nijel input from numeric pad always dropped when numlock off (2004-11-27) http://python.org/sf/1074333 closed by kbk mkarg undocumented (2004-11-24) http://python.org/sf/1072410 closed by rhettinger C:\Python24\Lib\compileall.py returns False (2004-11-19) http://python.org/sf/1069409 closed by kbk Comments regarding 'macintosh' behaviour wrong for MacOS X (2004-12-19) http://python.org/sf/1088119 closed by bcannon zlib decompressobj documentation typo (2004-12-19) http://python.org/sf/1088206 closed by rhettinger socket object method "makefile" has wrong doc (2003-11-03) http://python.org/sf/835300 closed by rhettinger calculation wrong rounding (2004-12-20) http://python.org/sf/1088563 closed by tim_one _sre.c references uninitialised memory (2004-12-21) http://python.org/sf/1088891 closed by niemeyer Incorrect behaviour of StreamReader.readline leads to crash (2004-12-01) http://python.org/sf/1076985 closed by doerwalter SSL-ed sockets don't close correct? (2004-06-24) http://python.org/sf/978833 closed by bcannon Carbon.File.FSCatalogInfo.createDate implementation (2004-12-22) http://python.org/sf/1089643 closed by ronaldoussoren Installation ends prematurely (2004-12-14) http://python.org/sf/1085208 closed by loewis python-2.4.msi install error (2004-12-05) http://python.org/sf/1079545 closed by loewis Windows 2.4c1 installer default location issues (2004-11-28) http://python.org/sf/1074873 closed by loewis Bad IDLE shortcut by 2.4 installer on XP (2004-12-01) http://python.org/sf/1076861 closed by loewis Uninstaller should restore file associations if possible (2004-12-14) http://python.org/sf/1085172 closed by loewis urllib2's HTTPPasswordMgr and uri's with port numbers (2004-06-17) http://python.org/sf/974757 closed by jhylton wininst --install-script leaves residual files on C: (2004-11-17) http://python.org/sf/1067732 closed by theller bdist_wininst improvements (2004-07-09) http://python.org/sf/988022 closed by theller Installing w/o admin generates key error (2002-08-27) http://python.org/sf/600952 closed by theller Windows installer halts (2003-11-25) http://python.org/sf/848871 closed by theller New / Reopened RFE __________________ long int bitwise ops speedup (patch included) (2004-12-18) http://python.org/sf/1087418 opened by Gregory Smith subclassable mmap (2004-12-18) http://python.org/sf/1087741 opened by Josiah Carlson optparse .error() should print options list (2004-12-22) http://python.org/sf/1089955 opened by Mike Orr From tcdelaney at optusnet.com.au Thu Dec 23 05:22:01 2004 From: tcdelaney at optusnet.com.au (Tim Delaney) Date: Thu Dec 23 05:22:09 2004 Subject: [Python-Dev] Python for Series 60 released References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com> <ca471dc2041222181945d20394@mail.gmail.com> Message-ID: <00ab01c4e8a6$f3bab050$4c00a8c0@ryoko> Guido van Rossum wrote: > Python runs on Nokia cell phones (the high-end ones, anyway) and has > support from Nokia! > > Pretty cool all around. I couldn't find out which version of Python is supported - have they told you? Cheers. Tim Delaney From bob at redivi.com Thu Dec 23 05:32:35 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Dec 23 05:32:41 2004 Subject: [Python-Dev] Python for Series 60 released In-Reply-To: <00ab01c4e8a6$f3bab050$4c00a8c0@ryoko> References: <AA11E1E902CA54468B88D06A86A81A4A0B4DA6@esebe056.ntc.nokia.com> <ca471dc2041222181945d20394@mail.gmail.com> <00ab01c4e8a6$f3bab050$4c00a8c0@ryoko> Message-ID: <AB94CAFC-549B-11D9-B983-000A9567635C@redivi.com> On Dec 22, 2004, at 11:22 PM, Tim Delaney wrote: > Guido van Rossum wrote: > >> Python runs on Nokia cell phones (the high-end ones, anyway) and has >> support from Nokia! >> >> Pretty cool all around. > > I couldn't find out which version of Python is supported - have they > told you? Python 2.2.2 (#0, Dec 2 2004, 18:12:32) [GCC 2.9-psion-98r2 (Symbian build 540)] on symbian_s60 Type "copyright", "credits" or "license" for more information. Type "commands" to see the commands available in this simple line editor. screenshot of a bluetooth serial session w/ the device: http://redivi.com/~bob/s60_python.?pdf This is on a Series 60 v1.0 device (3650, IIRC). -bob From firemoth at gmail.com Thu Dec 23 06:36:49 2004 From: firemoth at gmail.com (Timothy Fitz) Date: Thu Dec 23 06:38:44 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <20041219101324.GA17049@caltech.edu> References: <20041219101324.GA17049@caltech.edu> Message-ID: <972ec5bd04122221362a95fad7@mail.gmail.com> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than > float-->int. Permits larger floats (2.0**62) to match large > int (2**62) arguments. rhettinger marked as "won't fix" in > the original bug report; this seems like a clean solution, > tho. Recommend apply. Wouldn't this cause subtle errors when the float -> long conversion is no longer precise? Or is this a non issue because it could only happen when seeking on impossibly large files? From bob at redivi.com Thu Dec 23 07:18:33 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Dec 23 07:18:44 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <972ec5bd04122221362a95fad7@mail.gmail.com> References: <20041219101324.GA17049@caltech.edu> <972ec5bd04122221362a95fad7@mail.gmail.com> Message-ID: <78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com> On Dec 23, 2004, at 12:36 AM, Timothy Fitz wrote: >> 1067760 -- float-->long conversion on fileobj.seek calls, rather than >> float-->int. Permits larger floats (2.0**62) to match large >> int (2**62) arguments. rhettinger marked as "won't fix" in >> the original bug report; this seems like a clean solution, >> tho. Recommend apply. > > Wouldn't this cause subtle errors when the float -> long conversion is > no longer precise? Or is this a non issue because it could only happen > when seeking on impossibly large files? I think that Raymond marked as "won't fix" because automatic float -> integer conversion has been deprecated since Python 2.3.0 (if not earlier), for exactly the reason you state. It's dumb. >>> range(2.7) __main__:1: DeprecationWarning: integer argument expected, got float [0, 1] Apparently file.seek doesn't have this DeprecationWarning though.. Strange, that. >>> f.seek(3.6) >>> f.tell() 3L -bob From gvanrossum at gmail.com Thu Dec 23 07:39:57 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Dec 23 07:40:00 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com> References: <20041219101324.GA17049@caltech.edu> <972ec5bd04122221362a95fad7@mail.gmail.com> <78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com> Message-ID: <ca471dc20412222239326d0dda@mail.gmail.com> > Apparently file.seek doesn't have this DeprecationWarning though.. > Strange, that. > >>> f.seek(3.6) > >>> f.tell() > 3L That's a bug. Who'll fix it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at iinet.net.au Thu Dec 23 12:41:20 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Dec 23 12:41:25 2004 Subject: [Python-Dev] Website documentation - link to descriptor information Message-ID: <41CAAEE0.70301@iinet.net.au> I just spent 10 minutes hunting through the Python website for this link: http://www.python.org/doc/newstyle.html I knew it was there somewhere, I just couldn't find the darn thing. It turns out the major mistake I made was to start from "docs.python.org" instead of "www.python.org/doc". Would it be possible for the "New-style classes" link to be added to the sidebar menu for the individual version's documentation pages? Or else given its own link on the Topic Guides page? Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From titus at caltech.edu Thu Dec 23 06:51:33 2004 From: titus at caltech.edu (Titus Brown) Date: Thu Dec 23 16:37:38 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <972ec5bd04122221362a95fad7@mail.gmail.com> References: <20041219101324.GA17049@caltech.edu> <972ec5bd04122221362a95fad7@mail.gmail.com> Message-ID: <20041223055133.GB4353@caltech.edu> -> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than -> > float-->int. Permits larger floats (2.0**62) to match large -> > int (2**62) arguments. rhettinger marked as "won't fix" in -> > the original bug report; this seems like a clean solution, -> > tho. Recommend apply. -> -> Wouldn't this cause subtle errors when the float -> long conversion is -> no longer precise? Or is this a non issue because it could only happen -> when seeking on impossibly large files? When would the float --> long conversion not be precise? The docs say PyLong_FromDouble takes the integer part, so it's like doing a floor(), yes? The current behavior is to convert directly from float to int, even though seek() can take longs as an argument. Thus 2.0**31 works, 2**31 works, 2**62 works, but 2.0**62 doesn't. This seems mildly inconsistent and prompted the guy to submit a bug report, and then a patch. The patch (with a bit more code context) is: -- if (!PyArg_ParseTuple(args, "O|i:seek", &offobj, &whence)) return NULL; #if !defined(HAVE_LARGEFILE_SUPPORT) offset = PyInt_AsLong(offobj); #else + if(PyFloat_Check(offobj)) { + offobj = PyLong_FromDouble(PyFloat_AsDouble(offobj)); + } offset = PyLong_Check(offobj) ? PyLong_AsLongLong(offobj) : PyInt_AsLong(offobj); #endif -- so the issue only comes into play with large file support. Heh, and I just noticed that PyLong_FromDouble returns a new reference, so this is a memory leak, I believe. Whoooops. I'll fix that. cheers, --titus From titus at caltech.edu Thu Dec 23 09:28:19 2004 From: titus at caltech.edu (Titus Brown) Date: Thu Dec 23 16:37:40 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <ca471dc20412222239326d0dda@mail.gmail.com> References: <20041219101324.GA17049@caltech.edu> <972ec5bd04122221362a95fad7@mail.gmail.com> <78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com> <ca471dc20412222239326d0dda@mail.gmail.com> Message-ID: <20041223082819.GA2455@caltech.edu> -> > Apparently file.seek doesn't have this DeprecationWarning though.. -> > Strange, that. -> > >>> f.seek(3.6) -> > >>> f.tell() -> > 3L -> -> That's a bug. Who'll fix it? Added: + if (PyFloat_Check(offobj)) + PyErr_Warn(PyExc_DeprecationWarning, + "integer argument expected, got float" ); see attached diff. I also attached it to the patch at SourceForge, with a comment, just to keep the record straight. The DeprecationWarning wasn't given because a manual PyObject --> int/long conversion was being done, rather than the usual conversion implicit in ParseTuple. Regression tests run w/o problem on RH 9.0/i386. Now you get the correct behavior: -- >>> f = open('/dev/zero') >>> f.seek(5) >>> f.tell() 5L >>> f.seek(5.2) __main__:1: DeprecationWarning: integer argument expected, got float >>> f.tell() 5L -- This seems like a reasonable resolution to patch #1067760, to me... cheers, --titus -------------- next part -------------- ? Objects/.fileobject.c.swp Index: Objects/fileobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/fileobject.c,v retrieving revision 2.193 diff -c -c -r2.193 fileobject.c *** Objects/fileobject.c 7 Nov 2004 14:15:28 -0000 2.193 --- Objects/fileobject.c 23 Dec 2004 08:19:20 -0000 *************** *** 462,467 **** --- 462,472 ---- whence = 0; if (!PyArg_ParseTuple(args, "O|i:seek", &offobj, &whence)) return NULL; + + if (PyFloat_Check(offobj)) + PyErr_Warn(PyExc_DeprecationWarning, + "integer argument expected, got float" ); + #if !defined(HAVE_LARGEFILE_SUPPORT) offset = PyInt_AsLong(offobj); #else From mwh at python.net Thu Dec 23 17:48:17 2004 From: mwh at python.net (Michael Hudson) Date: Thu Dec 23 17:48:19 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <20041223055133.GB4353@caltech.edu> (Titus Brown's message of "Wed, 22 Dec 2004 21:51:33 -0800") References: <20041219101324.GA17049@caltech.edu> <972ec5bd04122221362a95fad7@mail.gmail.com> <20041223055133.GB4353@caltech.edu> Message-ID: <2mllbpj6zy.fsf@starship.python.net> Titus Brown <titus@caltech.edu> writes: > -> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than > -> > float-->int. Permits larger floats (2.0**62) to match large > -> > int (2**62) arguments. rhettinger marked as "won't fix" in > -> > the original bug report; this seems like a clean solution, > -> > tho. Recommend apply. > -> > -> Wouldn't this cause subtle errors when the float -> long conversion is > -> no longer precise? Or is this a non issue because it could only happen > -> when seeking on impossibly large files? > > When would the float --> long conversion not be precise? When the float is so large that it is the closest approximation to more than one integer? (i.e. larger than 2**53 for 754 doubles). Cheers, mwh -- #ifndef P_tmpdir printf( "Go buy a better computer" ); exit( ETHESKYISFALLINGANDIWANTMYMAMA ); -- Dimitri Maziuk on writing secure code, asr From mwh at python.net Thu Dec 23 17:50:35 2004 From: mwh at python.net (Michael Hudson) Date: Thu Dec 23 17:50:38 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <20041223082819.GA2455@caltech.edu> (Titus Brown's message of "Thu, 23 Dec 2004 00:28:19 -0800") References: <20041219101324.GA17049@caltech.edu> <972ec5bd04122221362a95fad7@mail.gmail.com> <78D0DFF7-54AA-11D9-B983-000A9567635C@redivi.com> <ca471dc20412222239326d0dda@mail.gmail.com> <20041223082819.GA2455@caltech.edu> Message-ID: <2mhdmdj6w4.fsf@starship.python.net> Titus Brown <titus@caltech.edu> writes: > -> > Apparently file.seek doesn't have this DeprecationWarning though.. > -> > Strange, that. > -> > >>> f.seek(3.6) > -> > >>> f.tell() > -> > 3L > -> > -> That's a bug. Who'll fix it? > > Added: > > + if (PyFloat_Check(offobj)) > + PyErr_Warn(PyExc_DeprecationWarning, > + "integer argument expected, got float" ); > > see attached diff. I also attached it to the patch at SourceForge, > with a comment, just to keep the record straight. You should check the return value of PyErr_Warn in case the user passed -Werror on the command line. > This seems like a reasonable resolution to patch #1067760, to me... I guess I could look at that -- but when I'm not on dialup at my parents' house... Cheers, mwh -- I love the way Microsoft follows standards. In much the same manner that fish follow migrating caribou. -- Paul Tomblin -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From jcarlson at uci.edu Thu Dec 23 17:43:53 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Dec 23 17:52:27 2004 Subject: [Python-Dev] Patches: 1 for the price of 10. In-Reply-To: <20041223055133.GB4353@caltech.edu> References: <972ec5bd04122221362a95fad7@mail.gmail.com> <20041223055133.GB4353@caltech.edu> Message-ID: <20041223084029.9246.JCARLSON@uci.edu> Titus Brown <titus@caltech.edu> wrote: > > -> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than > -> > float-->int. Permits larger floats (2.0**62) to match large > -> > int (2**62) arguments. rhettinger marked as "won't fix" in > -> > the original bug report; this seems like a clean solution, > -> > tho. Recommend apply. > -> > -> Wouldn't this cause subtle errors when the float -> long conversion is > -> no longer precise? Or is this a non issue because it could only happen > -> when seeking on impossibly large files? > > When would the float --> long conversion not be precise? The docs > say PyLong_FromDouble takes the integer part, so it's like doing > a floor(), yes? Precision. >>> 2.0**52 4503599627370496.0 >>> 2.0**52+1 4503599627370497.0 >>> 2.0**53 9007199254740992.0 >>> 2.0**53+1 9007199254740992.0 Anything above float(2**53-1) (one must be careful about the order of float conversions) are basically useless for offsets into files. > The current behavior is to convert directly from float to int, even > though seek() can take longs as an argument. Thus 2.0**31 works, > 2**31 works, 2**62 works, but 2.0**62 doesn't. This seems mildly > inconsistent and prompted the guy to submit a bug report, and then > a patch. "works" and "doesn't work" aren't precise. "doesn't raise an exception" and "does raise an exception" are precise. Note the above float precision precision example about why even non-exceptions may not "work". - Josiah From gvanrossum at gmail.com Thu Dec 23 18:53:47 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Dec 23 18:53:50 2004 Subject: [Python-Dev] Website documentation - link to descriptor information In-Reply-To: <41CAAEE0.70301@iinet.net.au> References: <41CAAEE0.70301@iinet.net.au> Message-ID: <ca471dc20412230953440f8eaf@mail.gmail.com> > It turns out the major mistake I made was to start from "docs.python.org" > instead of "www.python.org/doc". If you ask me that wasn't your mistake but the mistake of whoever decided to create a subdomain for docs (or for anything else). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pycon at python.org Thu Dec 23 22:25:55 2004 From: pycon at python.org (Steve Holden) Date: Thu Dec 23 22:25:56 2004 Subject: [Python-Dev] PyCon Registration Opens Today! Message-ID: <20041223212555.AAD4B1E400F@bag.python.org> Dear Python User: Following my last message, I am pleased to be able to announce that you can register for PyCon DC 2005 on the web at http://www.python.org/pycon/2005/register.html starting at 1700 EST today (December 23). Thanks to George Belotsky of Open Light Software for assistance in preparing the credit card processing software. As always, further information about PyCon is available in the usual places: http://www.pycon.org/ http://www.python.org/pycon/ I look forward to meeting you at PyCON DC 2005. In the meantime please let me offer my best wishes for a happy and peaceful holiday season. regards Steve Holden Chairman, PyCON DC 2005 -- PyCon DC 2005: The third Python Community Conference http://www.pycon.org/ http://www.python.org/pycon/ The scoop on Python implementations and applications From barry at barrys-emacs.org Thu Dec 23 23:20:33 2004 From: barry at barrys-emacs.org (Barry Scott) Date: Thu Dec 23 23:21:47 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? Message-ID: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> I see that, as expected, windows python 2.4 was built with MSVC 7.1 rather then msvc 6.0. It seems that I can build extensions with msvc 6.0 that work with the python 2.4 windows binary kit. Is this safe? I recall warning a while ago about mixing msvc 6.0 and msvc 7.1 runtime DLL's. Is this an issue with python 2.4? I'm also surprised that the python 2.4 source kit only mentions MSVC 6.0 and not the compiler that you actually built python 2.4 with. Barry From martin at v.loewis.de Fri Dec 24 00:12:42 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Dec 24 00:12:38 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? In-Reply-To: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> Message-ID: <41CB50EA.9070100@v.loewis.de> Barry Scott wrote: > It seems that I can build extensions with msvc 6.0 that work with the > python 2.4 windows > binary kit. > > Is this safe? No, it isn't. This emerges as a Python 2.4 FAQ. > I recall warning a while ago about mixing msvc 6.0 and msvc 7.1 runtime > DLL's. Is this > an issue with python 2.4? Yes, it is. > I'm also surprised that the python 2.4 source kit only mentions MSVC 6.0 > and not the compiler that you actually built python 2.4 with. Why do you say that? PCbuild/readme.txt starts with Building Python using VC++ 7.1 ------------------------------------- This directory is used to build Python for Win32 platforms, e.g. Windows 95, 98 and NT. It requires Microsoft Visual C++ 7.1 (a.k.a. Visual Studio .NET 2003). (For other Windows platforms and compilers, see ../PC/readme.txt.) Regards, Martin From barry at barrys-emacs.org Fri Dec 24 01:17:49 2004 From: barry at barrys-emacs.org (Barry Scott) Date: Fri Dec 24 01:19:06 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? In-Reply-To: <41CB50EA.9070100@v.loewis.de> References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> <41CB50EA.9070100@v.loewis.de> Message-ID: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> On Dec 23, 2004, at 23:12, Martin v. L?wis wrote: > Barry Scott wrote: >> It seems that I can build extensions with msvc 6.0 that work with the >> python 2.4 windows >> binary kit. >> Is this safe? > > No, it isn't. This emerges as a Python 2.4 FAQ. > >> I recall warning a while ago about mixing msvc 6.0 and msvc 7.1 >> runtime DLL's. Is this >> an issue with python 2.4? > > Yes, it is. > >> I'm also surprised that the python 2.4 source kit only mentions MSVC >> 6.0 and not the compiler that you actually built python 2.4 with. > > Why do you say that? PCbuild/readme.txt starts with I recursive grep'ed and missed this ref. However I did read this in README.TXT: > Building on non-UNIX systems > ---------------------------- > For Windows (2000/NT/ME/98/95), assuming you have MS VC++ 6.0, the > project files are in PCbuild, the workspace is pcbuild.dsw. See > PCbuild\readme.txt for detailed instructions. Which says that PCbuild/readme.txt is about VC++ 6.0 > Building Python using VC++ 7.1 > ------------------------------------- > This directory is used to build Python for Win32 platforms, e.g. > Windows > 95, 98 and NT. It requires Microsoft Visual C++ 7.1 > (a.k.a. Visual Studio .NET 2003). > (For other Windows platforms and compilers, see ../PC/readme.txt.) > > Regards, > Martin Barry From martin at v.loewis.de Fri Dec 24 09:29:13 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Dec 24 09:29:08 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? In-Reply-To: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> <41CB50EA.9070100@v.loewis.de> <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> Message-ID: <41CBD359.4020007@v.loewis.de> Barry Scott wrote: > I recursive grep'ed and missed this ref. However I did read this in > README.TXT: > > > Building on non-UNIX systems > > ---------------------------- > > > For Windows (2000/NT/ME/98/95), assuming you have MS VC++ 6.0, the > > project files are in PCbuild, the workspace is pcbuild.dsw. See > > PCbuild\readme.txt for detailed instructions. > > Which says that PCbuild/readme.txt is about VC++ 6.0 I see. This is now corrected in CVS. Regards, Martin From jlg at dds.nl Fri Dec 24 12:08:44 2004 From: jlg at dds.nl (Johannes Gijsbers) Date: Fri Dec 24 12:05:34 2004 Subject: [Python-Dev] Website documentation - link to descriptor information In-Reply-To: <41CAAEE0.70301@iinet.net.au> References: <41CAAEE0.70301@iinet.net.au> Message-ID: <20041224110844.GE9338@authsmtp.dds.nl> On Thu, Dec 23, 2004 at 09:41:20PM +1000, Nick Coghlan wrote: > I just spent 10 minutes hunting through the Python website for this link: > http://www.python.org/doc/newstyle.html > > I knew it was there somewhere, I just couldn't find the darn thing. > > It turns out the major mistake I made was to start from "docs.python.org" > instead of "www.python.org/doc". It's on docs.python.org now. Cheers, Johannes From mwh at python.net Fri Dec 24 12:37:23 2004 From: mwh at python.net (Michael Hudson) Date: Fri Dec 24 12:37:25 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? In-Reply-To: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> (Barry Scott's message of "Fri, 24 Dec 2004 00:17:49 +0000") References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> <41CB50EA.9070100@v.loewis.de> <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> Message-ID: <2m1xdgj5ak.fsf@starship.python.net> Barry Scott <barry@barrys-emacs.org> writes: > I recursive grep'ed and missed this ref. However I did read this in > README.TXT: The top-level README file is hilariously out-of-date, in some ways. I meant to do something about this before 2.4 final, but didn't get around to it... Cheers, mwh -- <exarkun> speak of the devil <moshez> exarkun: froor <exarkun> not you -- from Twisted.Quotes From arigo at tunes.org Fri Dec 24 13:32:27 2004 From: arigo at tunes.org (Armin Rigo) Date: Fri Dec 24 13:42:30 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? In-Reply-To: <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> <41CB50EA.9070100@v.loewis.de> <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> Message-ID: <20041224123227.GA12925@vicky.ecs.soton.ac.uk> Hi, On Fri, Dec 24, 2004 at 12:17:49AM +0000, Barry Scott wrote: > I recursive grep'ed and missed this ref. However I did read this in > README.TXT: The "extending and embedding" tutorial is similarily out-of-date. Armin From mwh at python.net Fri Dec 24 14:05:55 2004 From: mwh at python.net (Michael Hudson) Date: Fri Dec 24 14:05:57 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? In-Reply-To: <20041224123227.GA12925@vicky.ecs.soton.ac.uk> (Armin Rigo's message of "Fri, 24 Dec 2004 12:32:27 +0000") References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> <41CB50EA.9070100@v.loewis.de> <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> <20041224123227.GA12925@vicky.ecs.soton.ac.uk> Message-ID: <2mwtv7j170.fsf@starship.python.net> Armin Rigo <arigo@tunes.org> writes: > Hi, > > On Fri, Dec 24, 2004 at 12:17:49AM +0000, Barry Scott wrote: >> I recursive grep'ed and missed this ref. However I did read this in >> README.TXT: > > The "extending and embedding" tutorial is similarily out-of-date. Well, I've half re-written that (the entire thing, see doc-sig passim) but this hasn't attracted a great deal of interest. I've just uploaded my most recent version to: http://starship.python.net/crew/mwh/toext/ I haven't done any substantial work on it in a while, and probably won't have time to do any for some time. It would still be nice to have it ready in the 2.5 timeframe. Cheers, mwh -- A difference which makes no difference is no difference at all. -- William James (I think. Reference anyone?) From gvanrossum at gmail.com Fri Dec 24 16:39:06 2004 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri Dec 24 16:39:09 2004 Subject: [Python-Dev] Merry Christmas and a Happy New Year to all Python developers! Message-ID: <ca471dc2041224073912c09188@mail.gmail.com> I'm about to travel to a place where I don't expect to have internet access for over a week, so this is my last message in 2004. I'll have more time next year for Python that I had this year, so I'm looking forward to working again with this great community. I wish everyone happy celebrations of the dark season, and I hope to see many of you at the upcoming PyCon conference. Cheers! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at ocf.berkeley.edu Fri Dec 24 21:43:19 2004 From: bac at ocf.berkeley.edu (Brett C.) Date: Fri Dec 24 21:43:20 2004 Subject: [Python-Dev] proto-pep: How to change Python's bytecode Message-ID: <41CC7F67.9070009@ocf.berkeley.edu> After implementing over 10 new opcodes for my thesis I figured I should write down the basic steps in an info PEP so that there is enough guidelines with this PEP and PEP 306 to cover the bases on changes to the language itself. To go along with this I also plan to write some benchmarks for individual opcodes that could possibly lead to a testing suite for the opcodes themselves (will probably do this piece-meal and put it up on SF initially since there are a lot of opcodes). Anyway, let me know if I seem to be missing anything or have something to add. After a reasonable time of non-response to this I will request a PEP number (assuming people don't think this PEP is stupid). ------------------------------------------ PEP: XXX Title: How to change Python's bytecode Version: $Revision: 1.4 $ Last-Modified: $Date: 2003/09/22 04:51:50 $ Author: Brett Cannoon <brett@python.org> Status: Draft Type: Informational Content-Type: text/x-rst Created: XX-XXX-XXXX Post-History: XX-XXX-XXXX Abstract ======== Python source code is compiled down to something called bytecode. This bytecode (which can be viewed as sequences of opcodes) defines what Python is capable of. As such, knowing how to add, remove, or change the bytecode is important to do properly when changing the abilities of the Python language. Rationale ========= While changing Python's bytecode is not a frequent occurence, it still happens. Having the required steps documented in a single location should make experimentation with the bytecode easier since it is not necessarily obvious what the steps are to change the bytecode. This PEP, paired with PEP 306 [#PEP-306]_, should provide enough basic guidelines for handling any changes performed to the Python language itself in terms of syntactic changes that introduce new semantics. Checklist ========= This is a rough checklist of what files need to change and how they are involved with the bytecode. All paths are given from the viewpoint of ``/cvsroot/python/dist/src`` from CVS). This list should not be considered exhaustive nor to cover all possible situations. - ``Include/opcode.h`` This include file lists all known opcodes and associates each opcode name with a unique number. When adding a new opcode it is important to take note of the ``HAVE_ARGUMENT`` value. This ``#define``'s value specifies the value at which all opcodes that have a value greater than ``HAVE_ARGUMENT`` are expected to take an argument to the opcode. - ``Lib/opcode.py`` Lists all of the opcodes and their associated value. Used by the dis module [#dis]_ to map bytecode values to their names. - ``Python/ceval.c`` Contains the main interpreter loop. Code to handle the evalution of an opcode here. - ``Python/compile.c`` To make sure an opcode is actually used, this file must be altered. The emitting of all bytecode occurs here. - ``Lib/compiler/pyassem.py``, ``Lib/compiler/pycodegen.py`` The 'compiler' package [#compiler]_ needs to be altered to also reflect any changes to the bytecode. - ``Doc/lib/libdis.tex`` The documentation [#dis-docs] for the dis module contains a complete list of all the opcodes. - ``Python/import.c`` Defines the magic word (named ``MAGIC``) used in .pyc files to detect if the bytecode used matches the one used by the version of Python running. This number needs to be changed to make sure that the running interpreter does not try to execute bytecode that it does not know about. Suggestions for bytecode development ==================================== A few things can be done to make sure that development goes smoothly when experimenting with Python's bytecode. One is to delete all .py(c|o|w) files after each semantic change to Python/compile.c . That way all files will use any bytecode changes. Make sure to run the entire testing suite [#test-suite]_. Since the ``regrtest.py`` driver recompiles all source code before a test is run it acts a good test to make sure that no existing semantics are broken. Running parrotbench [#parrotbench]_ is also a good way to make sure existing semantics are not broken; this benchmark is practically a compliance test. References ========== .. [#PEP-306] PEP 306, How to Change Python's Grammar, Hudson (http://www.python.org/peps/pep-0306.html) .. [#dis] XXX .. [#test-suite] XXX .. [#parrotbench] XXX .. [#dis-docs] XXX Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From Scott.Daniels at Acm.Org Sat Dec 25 02:04:38 2004 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sat Dec 25 02:03:15 2004 Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode In-Reply-To: <41CC7F67.9070009@ocf.berkeley.edu> References: <41CC7F67.9070009@ocf.berkeley.edu> Message-ID: <cqie8c$jv$1@sea.gmane.org> Brett C. wrote: > A few things can be done to make sure that development goes smoothly when > experimenting with Python's bytecode. One is to delete all .py(c|o|w) Don't you mean ".pyc or .pyo (remember such files in zips as well)" .pyw is normal python source. -- Scott David Daniels Scott.Daniels@Acm.Org From tjreedy at udel.edu Sat Dec 25 02:54:32 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Dec 25 02:54:37 2004 Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode References: <41CC7F67.9070009@ocf.berkeley.edu> Message-ID: <cqih8n$4du$1@sea.gmane.org> "Brett C." <bac@ocf.berkeley.edu> wrote in message news:41CC7F67.9070009@ocf.berkeley.edu... At to the title, bytecodes are a property of the CPython implementation, not of Python itself. Since I think the distinction is quite important to maintain, I would insert the missing 'C' and everywhere else as appropriate. > After implementing over 10 new opcodes for my thesis I figured I should > write down the basic steps in an info PEP so that there is enough > guidelines with this PEP and PEP 306 to cover the bases on changes to the > language itself. Over the last several years, various people have reported experimenting with CPython's bytecodes. I wonder if it would be helpful to have a respository of the results, in one place, for new experimenters and curious people to peruse. > Anyway, let me know if I seem to be missing anything or have something to > add. As I said, the importanct 'C' qualifier. > PEP: XXX > Title: How to change Python's bytecode /P/CP/ > Python source code is compiled down to something called bytecode. Suggested replacements in quotes: "CPython compiles Python source code to something called bytecode." > This bytecode (which can be viewed as sequences of opcodes) > defines what Python is capable of. This is backwards. "The language, as defined in the Reference Manual, determines what bytecodes are needed (collectively, not one by one) for a bytecode implementation." > As such, knowing how to add, remove, or change the bytecode is > important to do properly when changing the abilities of the Python > language. "Therefore, changes in the language may require changes in the set of bytecodes. In addition, changing the bytecode set for a given definition may result in desireable changes in the interpreter behavior. This document describes how to do so for either reason." > Rationale > ========= > > While changing Python's bytecode is not a frequent occurence, it still > happens. /P/CP/ Experiments are much more frequent than committed changes -- all of which start as experiments. ... Terry J. Reedy From bac at ocf.berkeley.edu Sat Dec 25 20:31:06 2004 From: bac at ocf.berkeley.edu (Brett C.) Date: Sat Dec 25 20:31:13 2004 Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode In-Reply-To: <cqie8c$jv$1@sea.gmane.org> References: <41CC7F67.9070009@ocf.berkeley.edu> <cqie8c$jv$1@sea.gmane.org> Message-ID: <41CDBFFA.1060405@ocf.berkeley.edu> Scott David Daniels wrote: > Brett C. wrote: > >> A few things can be done to make sure that development goes smoothly when >> experimenting with Python's bytecode. One is to delete all .py(c|o|w) > > Don't you mean ".pyc or .pyo (remember such files in zips as well)" > .pyw is normal python source. > Yeah, my bad. Been a while since I have written a .pyw file. -Brett From bac at ocf.berkeley.edu Sat Dec 25 20:36:17 2004 From: bac at ocf.berkeley.edu (Brett C.) Date: Sat Dec 25 20:36:17 2004 Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode In-Reply-To: <cqih8n$4du$1@sea.gmane.org> References: <41CC7F67.9070009@ocf.berkeley.edu> <cqih8n$4du$1@sea.gmane.org> Message-ID: <41CDC131.70702@ocf.berkeley.edu> Terry Reedy wrote: > "Brett C." <bac@ocf.berkeley.edu> wrote in message > news:41CC7F67.9070009@ocf.berkeley.edu... > > At to the title, bytecodes are a property of the CPython implementation, > not of Python itself. Since I think the distinction is quite important to > maintain, I would insert the missing 'C' and everywhere else as > appropriate. > Then the same could be said for PEP 306. But I don't mind changing the title. > >>After implementing over 10 new opcodes for my thesis I figured I should >>write down the basic steps in an info PEP so that there is enough >>guidelines with this PEP and PEP 306 to cover the bases on changes to the >>language itself. > > > Over the last several years, various people have reported experimenting > with CPython's bytecodes. I wonder if it would be helpful to have a > respository of the results, in one place, for new experimenters and curious > people to peruse. > Wouldn't hurt. Adding that section would not be bad, but I don't have the inclination to hunt them down. What do others think about having this section? -Brett From arigo at tunes.org Sat Dec 25 21:23:03 2004 From: arigo at tunes.org (Armin Rigo) Date: Sat Dec 25 21:33:07 2004 Subject: [Python-Dev] Upcoming PyPy sprint Message-ID: <20041225202303.GA16023@vicky.ecs.soton.ac.uk> Hi Python people, The next sprint for PyPy, the Python-in-Python interpreter, will take place in Leysin, in the lower mountains of Switzerland, 22nd - 29th January 2005 (travel days: 22nd and 29-30th) http://codespeak.net/moin/pypy/moin.cgi/LeysinSprint The technical goals will be two-fold: we want to continue the hard work started at the previous Vilnius sprint (a first generated C version of our interpreter); but the upcoming sprint will also be a "learning PyPy" sprint, covering all aspects of PyPy that are easier to start with. Some examples are described below. For more information about PyPy, see http://codespeak.net/pypy/index.cgi?doc . The "learning PyPy" focus comes from the fact that it will be our first sprint as a European Union sponsored group, and not all members of this EU/PyPy project are already familiar with PyPy. This is a good occasion for newcomers that want to look at PyPy more closely too, in a "fun and somewhat mind-altering" sprint event. Note that as part of our funding we want to be able to support some of the travel and lodging costs for a number of outside people as well, but although the corresponding money is in the budgets, not all administrative issues have been solved yet. We don't know yet if we will be able to do so for the January sprint. However, if you would like to come to the sprint but can't afford travel and accomodation costs then please contact us. Even if the administrative issues have not been sorted out, we may be able to help with private money. Just contact us or me personally for this sprint. About Leysin ------------ The place where the sprint will take place is located in the pre-Alps of the french-speaking (west) region of Switzerland. Leysin is a village at an altitude of 1200 to 1400m. It is a skiing station, and also the place where I (Armin) lived for 20 years. There are 2000 people in summer and 10000 in winter :-) but the sprint will be just before the most crowded periods of February. Of course one full day will be dedicated to winter sports! Both the sprint venue and the logding will be in a very spacious pair of chalets built specifically for bed&breakfast: http://www.ermina.ch/. The place has a baseline ADSL Internet connexion (600Kb); we need to arrange between ourselves to bring a wireless hub. You can of course arrange your own lodging anywhere (you cannot be more than a 15 minutes walk away from the sprint venue), but I definitely recommend to lodge there too -- you won't find better sights anywhere else (though you probably won't get much worse ones easily, either :-) Subscription ------------ Please subscribe at http://codespeak.net/moin/pypy/moin.cgi/LeysinSprintAttendants and mention if you would like to participate in the group reservation, and which size of room you would prefer (2 to 6 beds available). If you wish, you can extend your stay for some days after the 29th of January -- please mention it if you want to book with the group (but you cannot arrive there earier than the 22nd: fully booked). If you have any question don't hesitate to contact pypy-sprint@codespeak.net or one of us personally. Learning PyPy ------------- Here are some goals that are quite reachable without in-depth prior knowledge of PyPy: * systematic testing: the interpreter and type implementations work generally well, but there are quite some areas that need refactoring and refinement. A good approach is to run as many of CPython's tests as possible and fix stuff accordingly. This is a good way to learn random parts of PyPy until you know them all! * extension modules and extension types: some can be just rewritten as a plain Python equivalent, but others need to be more integrated into PyPy's internals. * the bytecode compiler: we don't have any compiler integrated with PyPy so far, but there exists a complete set of Python code out there (tokenizer, parser, st->ast, ast->bytecode) that could be used. We could also discuss ideas like syntax extensions generating new bytecodes, etc. * tracing and other wrappers: e.g. how to write a fully debugging Object Space that records the history of all changes to objects, etc. * more bits and pieces: small things missing from our interpreter to be fully compliant. Tracing/profiling hooks come to mind; more small things are listed in http://codespeak.net/svn/pypy/trunk/src/pypy/TODO . In addition to the above examples, there are a number of more involved tasks that nevertheless don't require a complete grasp of PyPy -- whose parts are relatively independent from each other. The "hard" work currently going on in PyPy is on the translation part, which is needed to make PyPy run faster and/or in other environments. This work is however mostly independent. For people that prefer to focus on cross-language stuff rather than Python internals we can discuss about tasks like writing the basics of an interpreter for another language -- for example, it would be great to have a decent error-checking Javascript interpreter, but we can discuss any other language, Python-like or not. In parallel, there is always pending work to allow the code generator to target other languages and platforms (currently there is C, Lisp, Pyrex and Java) or generate code differently (Stackless-style, Psyco-style). A bientot, Armin Rigo From Scott.Daniels at Acm.Org Sun Dec 26 01:37:40 2004 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun Dec 26 01:36:19 2004 Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode In-Reply-To: <41CDC131.70702@ocf.berkeley.edu> References: <41CC7F67.9070009@ocf.berkeley.edu> <cqih8n$4du$1@sea.gmane.org> <41CDC131.70702@ocf.berkeley.edu> Message-ID: <cql11r$ot$1@sea.gmane.org> Brett C. wrote: > Terry Reedy wrote: >> "Brett C." <bac@ocf.berkeley.edu> wrote.... >> Over the last several years, various people have reported >> experimenting with CPython's bytecodes. I wonder if it would be >> helpful to have a respository of the results, in one place, for new >> experimenters and curious people to peruse. > > Wouldn't hurt. Adding that section would not be bad, but I don't have > the inclination to hunt them down. What do others think about having > this section? What you could do is declare a wiki page which anyone could go to; less work for you, and forms a new place to pool knowledge. -- Scott David Daniels Scott.Daniels@Acm.Org From skip at pobox.com Sun Dec 26 01:43:31 2004 From: skip at pobox.com (Skip Montanaro) Date: Sun Dec 26 01:43:48 2004 Subject: [Python-Dev] Re: proto-pep: How to change Python's bytecode In-Reply-To: <41CDC131.70702@ocf.berkeley.edu> References: <41CC7F67.9070009@ocf.berkeley.edu> <cqih8n$4du$1@sea.gmane.org> <41CDC131.70702@ocf.berkeley.edu> Message-ID: <16846.2355.592620.198686@montanaro.dyndns.org> >> Over the last several years, various people have reported >> experimenting with CPython's bytecodes. I wonder if it would be >> helpful to have a respository of the results, in one place, for new >> experimenters and curious people to peruse. Brett> Wouldn't hurt. Adding that section would not be bad, but I don't Brett> have the inclination to hunt them down. What do others think Brett> about having this section? How about just references in the PEP? I presented a paper several years ago at a Python workshop on a peephole optimizer for Python. Also, Michael Hudson has his no-longer-active bytecodehacks stuff: http://www.foretec.com/python/workshops/1998-11/proceedings/papers/montanaro/montanaro.html http://bytecodehacks.sourceforge.net/bch-docs/bch/index.html I imagine there's other stuff as well. Skip From Jack.Jansen at cwi.nl Mon Dec 27 15:15:26 2004 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Mon Dec 27 15:15:32 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src setup.py, 1.208, 1.209 In-Reply-To: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net> References: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net> Message-ID: <C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl> On 18-dec-04, at 21:48, bcannon@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21493 > > Modified Files: > setup.py > Log Message: > Switch from getting LDFLAGS and CPPFLAGS from the environment to the > Makefile. > This is to avoid a problem that inconsistently comes up where the > environment > variable is unset while the Makefile clearly has the values set and > are used > during ``make``. Brett, I'm building MacPython binary distributions with an LDFLAGS configure option, as in $PYTHONSRC/configure -C --enable-framework LDFLAGS=-Wl,-x This has suddenly started failing with a very mysterious error message. When the make tries to build the extension modules I get running build running build_ext usage: setup.py [options] setup.py: error: no such option: -W Could this somehow be caused by your fix? -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From eric.nieuwland at xs4all.nl Mon Dec 27 22:30:32 2004 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Mon Dec 27 22:30:36 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <41D0542E.9000404@noaa.gov> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> Message-ID: <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> Hi all, On 27 dec 2004, at 19:27, Chris Barker wrote: > Robin has added versioning to the latest wxPython, and I. for one am > ecstatic. It works great for me. I am generally using 2.5.3, but have > 2.4.2 installed, and a number of my apps depend on it (on Linux > anyway, it's pretty useless on OS-X) > > It's great to be able to put: > > import wxversion > wxversion.select("2.4") > > At the top of my apps, and know that they'll use the version of > wxPython I tested against. Would it be an idea to submit a PEP for extending the 'import' keyword? I can imagine it to be like: import spam version 1 import foo version 2, 4 import bar version 7, 1, 6 with version numbers consisting of some fixed maximum of parts and the version number parts not given might be wildcards. We could then have a per module version system like: .../ spam/ 1.0.1/ ... [version 1.0.1 stuff in here] 1.5.7/ ... [version 1.5.7 stuff in here] there should be nothing else in the module directory, except for a mechanism to further support versioning. Default behaviour of 'import spam' would be to select the most recent version. We might want to override that by pointing to some other version. This might be achieved using a symbolic link/alias/shortcut to the directory of that version (spam/current -> spam/1.0.1). More like the directory/module method would be to allow for a __version__.py file in 'spam'. Content of this file is to be determined, but I think it should at least something like __current__ or __version__ to act as the pointer to the current version. --eric From martin at v.loewis.de Tue Dec 28 00:34:54 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Dec 28 00:34:57 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> Message-ID: <41D09C1E.1060009@v.loewis.de> Eric Nieuwland wrote: > Would it be an idea to submit a PEP for extending the 'import' keyword? No. Normally, packages should aim for backwards compatibility, so that applications would only want to specify a minimum version, such as import xml assert xml.version_info > (0,8,2) If you really want side-by-side installation of different versions, and a mechanism to select between them, the package could support import xml_0_8_2 as xml IOW, "import-as" should be sufficient for what you want to achieve. Regards, Martin From Scott.Daniels at Acm.Org Tue Dec 28 01:18:46 2004 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Tue Dec 28 01:17:22 2004 Subject: [Python-Dev] Zipfile needs? Message-ID: <cqq8mc$3g9$1@sea.gmane.org> I'm hoping to add BZIP2 compression to zipfile for 2.5. My primary motivation is that Project Gutenberg seems to be starting to use BZIP2 compression for some of its zips. What other wish list things do people around here have for zipfile? I thought I'd collect input here and make a PEP. I can open a pseudo-file for STORED files in binary read mode, for example, to allow reading zip-in-zip files without fully occupying memory. -- Scott David Daniels Scott.Daniels@Acm.Org From jcarlson at uci.edu Tue Dec 28 02:43:15 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue Dec 28 03:11:59 2004 Subject: [Python-Dev] Zipfile needs? In-Reply-To: <cqq8mc$3g9$1@sea.gmane.org> References: <cqq8mc$3g9$1@sea.gmane.org> Message-ID: <20041227174248.9252.JCARLSON@uci.edu> Scott David Daniels <Scott.Daniels@Acm.Org> wrote: > > I'm hoping to add BZIP2 compression to zipfile for 2.5. My primary > motivation is that Project Gutenberg seems to be starting to use BZIP2 > compression for some of its zips. What other wish list things do > people around here have for zipfile? I thought I'd collect input here > and make a PEP. > > I can open a pseudo-file for STORED files in binary read mode, for > example, to allow reading zip-in-zip files without fully occupying > memory. I'm not sure that zipfile needs BZIP2 support...being that there is a bzip2 module. - Josiah From bob at redivi.com Tue Dec 28 03:37:42 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Dec 28 03:37:53 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <41D0BAB4.70903@noaa.gov> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> <41D09C1E.1060009@v.loewis.de> <41D0BAB4.70903@noaa.gov> Message-ID: <729AC65F-5879-11D9-AA13-000A9567635C@redivi.com> On Dec 27, 2004, at 8:45 PM, Chris Barker wrote: > The versioning system that wxPython now has is quite nice, and seems > to fit most people's needs well. However, it's also quite new, and who > know what problems will arise. For those interested, here's a > synopsis. > > http://wiki.wxpython.org/index.cgi/MultiVersionInstalls I just cleaned up a bunch of spelling on that wiki page and rewrote the "what about py2exe" section. It now mentions py2app, is (hopefully) written more clearly, and includes an easier alternative for bundling the desired version of wxPython (wxversion can be used from setup.py). -bob From bob at redivi.com Tue Dec 28 03:41:23 2004 From: bob at redivi.com (Bob Ippolito) Date: Tue Dec 28 03:41:29 2004 Subject: [Python-Dev] Zipfile needs? In-Reply-To: <20041227174248.9252.JCARLSON@uci.edu> References: <cqq8mc$3g9$1@sea.gmane.org> <20041227174248.9252.JCARLSON@uci.edu> Message-ID: <F6A8BE19-5879-11D9-AA13-000A9567635C@redivi.com> On Dec 27, 2004, at 8:43 PM, Josiah Carlson wrote: > Scott David Daniels <Scott.Daniels@Acm.Org> wrote: >> >> I'm hoping to add BZIP2 compression to zipfile for 2.5. My primary >> motivation is that Project Gutenberg seems to be starting to use BZIP2 >> compression for some of its zips. What other wish list things do >> people around here have for zipfile? I thought I'd collect input here >> and make a PEP. >> >> I can open a pseudo-file for STORED files in binary read mode, for >> example, to allow reading zip-in-zip files without fully occupying >> memory. > > I'm not sure that zipfile needs BZIP2 support...being that there is a > bzip2 module. Note that the bzip2 module is named bz2 and does provide a file-like-interface. Also, it is implemented entirely as a C extension (ick). -bob From skip at pobox.com Tue Dec 28 05:26:51 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Dec 28 05:27:16 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <41D09C1E.1060009@v.loewis.de> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> <41D09C1E.1060009@v.loewis.de> Message-ID: <16848.57483.204434.666107@montanaro.dyndns.org> Martin> If you really want side-by-side installation of different Martin> versions, and a mechanism to select between them, the package Martin> could support Martin> import xml_0_8_2 as xml Martin> IOW, "import-as" should be sufficient for what you want to achieve. That's more-or-less the scheme adopted where I work. If we have two versions of a sybase module installed side-by-side, the imports might look like: import local.db.sybase.v1 as sybase or import local.db.sybase.v2 as sybase It's a bit cumbersome, but it's worked okay for us so far. Skip From Chris.Barker at noaa.gov Tue Dec 28 02:45:24 2004 From: Chris.Barker at noaa.gov (Chris Barker) Date: Tue Dec 28 05:27:35 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <41D09C1E.1060009@v.loewis.de> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> <41D09C1E.1060009@v.loewis.de> Message-ID: <41D0BAB4.70903@noaa.gov> Martin v. L?wis wrote: > No. Normally, packages should aim for backwards compatibility, so that > applications would only want to specify a minimum version, such as > > import xml > assert xml.version_info > (0,8,2) Well, yes, but life is not always so simple, and while, as a rule, version 2.3 should be backward compatible with 2.2, I see no reason to expect that version 5.0 and version 2.0 will be compatible. Frankly, backward compatibility can be a real impediment to progress. (when will "true division" be default?) > If you really want side-by-side installation of different versions, > and a mechanism to select between them, the package could support > > import xml_0_8_2 as xml > > IOW, "import-as" should be sufficient for what you want to achieve. This really doesn't work well for complex packages. I was quite involved with the debate about versioning for wxPython (and helped drive it happening) and that was what I originally proposed. The problem is that you have a whole pile of modules and libs and user code that all imports the package. There are a LOT of "import wx" lines in the wxPython library, and a whole bunch more in a sizable wxPython app. As you upgrade, every one of those would have to be changed to import wx_x_y_z as wx This was not considered a reasonable solution. Among other things, it's really nice to be able to make a small change in just one file, and be able to test your app against a new version. Other approaches were suggested and were used in the past: - A script that you run to change what version is installed - An environment variable that you set. - Whatever else I can't recall. Personally, I was only going to be really happy with an approach that worked at the python level, in one place. The versioning system that wxPython now has is quite nice, and seems to fit most people's needs well. However, it's also quite new, and who know what problems will arise. For those interested, here's a synopsis. http://wiki.wxpython.org/index.cgi/MultiVersionInstalls Skip Montanaro wrote: > There's more to it than that: > 1. Most people simply don't need it. My first brush with versioning > Python modules occurred in the past few months at my current job, and > that was mostly so our SWIG'd modules could be released in step with > our locally written C++ libraries. I used Python for ten years > before that without ever feeling the need for versioning. I'm surprised, but I think maturity and complexity has something to do with it. The more mature Python and its add-on packages become, the more this will be an issue. There was certainly a lot of uproar about backward compatible changes in Python itself, another issue that could be lessoned by a bit easier accommodation of multiple versions of python. (or why, or why did Redhat not put a version on their #! lines? have they yet?) > 2. I think it will be challenging to come up with a versioning scheme > that works for everyone. This is very true we had a hard enough time coming to a consensus among a small group of wxPython developers. I'm not sure we even did anyway, Robin just decided on one to implement. > To do versioning at work we had to solve > issues related to module naming, directory structure, source code > control and local build environment tools to make it work. Wow! that sounds a lot more complex that it has to be, but I'm sure there's a lot to what you've done. Note that you've kind of contradicted yourself (or at least contradicted Martin). I suspect your versioning issues would have been much simpler if Python itself had supported it. > Write a PEP and put it out for review. I don't recall seeing this raised in > the Python community before. That I'm surprised about. I've seen it (and brought it up) a number of times. I know there are a few major packages with roll your own versioning systems, wxPython, PyGTK, and PMW to mention a few. > I certainly don't think it's something the > core developers worry about, This is quite true, and why I haven't bothered with a PEP, but maybe I've got the thinking backwards, we need the PEP to get the "important" people thinking about it. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov From Scott.Daniels at Acm.Org Tue Dec 28 06:36:07 2004 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Tue Dec 28 06:34:43 2004 Subject: [Python-Dev] Re: Zipfile needs? In-Reply-To: <20041227174248.9252.JCARLSON@uci.edu> References: <cqq8mc$3g9$1@sea.gmane.org> <20041227174248.9252.JCARLSON@uci.edu> Message-ID: <cqqr9d$vrk$1@sea.gmane.org> Josiah Carlson wrote: > Scott David Daniels <Scott.Daniels@Acm.Org> wrote: > >>I'm hoping to add BZIP2 compression to zipfile for 2.5. My primary >>motivation is that Project Gutenberg seems to be starting to use BZIP2 >>compression for some of its zips. What other wish list things do >>people around here have for zipfile? I thought I'd collect input here >>and make a PEP. >> >>I can open a pseudo-file for STORED files in binary read mode, for >>example, to allow reading zip-in-zip files without fully occupying >>memory. > > > I'm not sure that zipfile needs BZIP2 support...being that there is a > bzip2 module. > > - Josiah > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > But if you look at the zipfile document, BZIP2 is a compression technique you can use (per file) in a zip archive. In fact, I use bz2 to compress/decompress, but the data still needs to inhabit the archive. -- -- Scott David Daniels Scott.Daniels@Acm.Org From bac at ocf.berkeley.edu Tue Dec 28 08:03:44 2004 From: bac at ocf.berkeley.edu (Brett C.) Date: Tue Dec 28 08:03:40 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src setup.py, 1.208, 1.209 In-Reply-To: <C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl> References: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net> <C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl> Message-ID: <41D10550.5000105@ocf.berkeley.edu> Jack Jansen wrote: > > On 18-dec-04, at 21:48, bcannon@users.sourceforge.net wrote: > >> Update of /cvsroot/python/python/dist/src >> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21493 >> >> Modified Files: >> setup.py >> Log Message: >> Switch from getting LDFLAGS and CPPFLAGS from the environment to the >> Makefile. >> This is to avoid a problem that inconsistently comes up where the >> environment >> variable is unset while the Makefile clearly has the values set and >> are used >> during ``make``. > > > Brett, > I'm building MacPython binary distributions with an LDFLAGS configure > option, as in > $PYTHONSRC/configure -C --enable-framework LDFLAGS=-Wl,-x > > This has suddenly started failing with a very mysterious error message. > When the > make tries to build the extension modules I get > running build > running build_ext > usage: setup.py [options] > setup.py: error: no such option: -W > > Could this somehow be caused by your fix? Yep, it's my doing. I bet optparse is raising an error since it doesn't have options for -W1 or -x registered with the parser. Anyone off the top of their head know how to cause optparse to just skip over options it doesn't know about? I am on vacation at the moment so I will deal with this when I get back this weekend. -Brett From eric.nieuwland at xs4all.nl Tue Dec 28 09:22:08 2004 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Tue Dec 28 09:22:11 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <41D09C1E.1060009@v.loewis.de> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> <41D09C1E.1060009@v.loewis.de> Message-ID: <90D320DB-58A9-11D9-8852-000393894CEA@xs4all.nl> Martin v. L?wis wrote: > Eric Nieuwland wrote: >> Would it be an idea to submit a PEP for extending the 'import' >> keyword? > > No. Normally, packages should aim for backwards compatibility, so that > applications would only want to specify a minimum version, such as > > import xml > assert xml.version_info > (0,8,2) > > If you really want side-by-side installation of different versions, > and a mechanism to select between them, the package could support > > import xml_0_8_2 as xml > > IOW, "import-as" should be sufficient for what you want to achieve. Unless you are doing comparison tests, where it would be nice to be able to state in a generic way that the new implementation should not change answers. May be something like: import spam[1] as spamnext # next version import spam[0] as spamnow # current version assert spamnow.Ni() == spamnext.Ni() --eric From skip at pobox.com Tue Dec 28 14:23:59 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Dec 28 14:24:04 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <90D320DB-58A9-11D9-8852-000393894CEA@xs4all.nl> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> <41D09C1E.1060009@v.loewis.de> <90D320DB-58A9-11D9-8852-000393894CEA@xs4all.nl> Message-ID: <16849.24175.124239.931764@montanaro.dyndns.org> Eric> Unless you are doing comparison tests, where it would be nice to Eric> be able to state in a generic way that the new implementation Eric> should not change answers. May be something like: Eric> import spam[1] as spamnext # next version Eric> import spam[0] as spamnow # current version Eric> assert spamnow.Ni() == spamnext.Ni() >From the Zen of Python I quote: Namespaces are one honking great idea -- let's do more of those! import spam.v1 as spamnext import spam.v0 as spamnow assert spamnow.Ni() == spamnext.Ni() Skip From martin at v.loewis.de Tue Dec 28 14:30:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Dec 28 14:30:49 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] The versioning question... In-Reply-To: <41D0BAB4.70903@noaa.gov> References: <a05200f02bdec9add0045@[81.107.210.110]> <8128F38C-686F-45F7-90C4-CB181B992AD1@mac.com> <a05200f01bdf0b1aae35e@[81.107.210.110]> <C804CB62-5522-11D9-B983-000A9567635C@redivi.com> <a05200f01bdf486f9ef33@[82.3.232.103]> <4520D194-5773-11D9-97BB-000A9567635C@redivi.com> <41D0542E.9000404@noaa.gov> <897942DB-584E-11D9-8852-000393894CEA@xs4all.nl> <41D09C1E.1060009@v.loewis.de> <41D0BAB4.70903@noaa.gov> Message-ID: <41D16006.8020103@v.loewis.de> Chris Barker wrote: > This really doesn't work well for complex packages. I was quite involved > with the debate about versioning for wxPython (and helped drive it > happening) and that was what I originally proposed. The problem is that > you have a whole pile of modules and libs and user code that all imports > the package. There are a LOT of "import wx" lines in the wxPython > library, and a whole bunch more in a sizable wxPython app. As you > upgrade, every one of those would have to be changed to > > import wx_x_y_z as wx Not at all. Import wx_x_y_z could do sys.modules["wx"] = wx_x_y_z and then all references to import wx would automatically resolve to the version that was imported first (or last, depending on your implementation strategy). > This was not considered a reasonable solution. Among other things, it's > really nice to be able to make a small change in just one file, and be > able to test your app against a new version. Even if the package is not prepared to register itself under a different name also, this one file could modify sys.modules. Or you can have a wx.py that reads from wx_x_y_z import * Then, wx.py would be that single file. > The versioning system that wxPython now has is quite nice, and seems to > fit most people's needs well. Very good! Then I don't see a need to change anything in Python. > This is quite true, and why I haven't bothered with a PEP, but maybe > I've got the thinking backwards, we need the PEP to get the "important" > people thinking about it. I don't think a PEP is needed - Python already appears to have everything you need to come up with your own versioning rules. Regards, Martin From martin at v.loewis.de Tue Dec 28 14:41:52 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Dec 28 14:41:55 2004 Subject: [Python-Dev] Zipfile needs? In-Reply-To: <cqq8mc$3g9$1@sea.gmane.org> References: <cqq8mc$3g9$1@sea.gmane.org> Message-ID: <41D162A0.9000502@v.loewis.de> Scott David Daniels wrote: > I'm hoping to add BZIP2 compression to zipfile for 2.5. My primary > motivation is that Project Gutenberg seems to be starting to use BZIP2 > compression for some of its zips. What other wish list things do > people around here have for zipfile? I thought I'd collect input here > and make a PEP. AFAIR, compression mechanisms are defined by numbers in the zip file. So you should not bother with such a change unless there is some "official" specification that explains how bzip2 is used in zipfiles. IOW, looking at http://www.pkware.com/company/standards/appnote/ you'll see that PKWARE has assigned algorithm 12 for bzip2. You might want to take a look at the spec to see what else the Python implementation lacks, and either document these features as deliberately missing, TODO, or just implement them right away. Regards, Martin From bac at ocf.berkeley.edu Tue Dec 28 20:15:18 2004 From: bac at ocf.berkeley.edu (Brett C.) Date: Tue Dec 28 20:15:19 2004 Subject: [Python-Dev] Zipfile needs? In-Reply-To: <cqq8mc$3g9$1@sea.gmane.org> References: <cqq8mc$3g9$1@sea.gmane.org> Message-ID: <41D1B0C6.8040208@ocf.berkeley.edu> Scott David Daniels wrote: > I'm hoping to add BZIP2 compression to zipfile for 2.5. My primary > motivation is that Project Gutenberg seems to be starting to use BZIP2 > compression for some of its zips. What other wish list things do > people around here have for zipfile? I thought I'd collect input here > and make a PEP. > Encryption/decryption support. Will most likely require a C extension since the algorithm relies on ints (or longs, don't remember) wrapping around when the value becomes too large. -Brett From bob at redivi.com Wed Dec 29 04:35:15 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Dec 29 04:35:21 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Mac/OSX fixapplepython23.py, 1.1, 1.2 In-Reply-To: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net> References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net> Message-ID: <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com> On Dec 28, 2004, at 4:30 PM, jackjansen@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Mac/OSX > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9229 > > Modified Files: > fixapplepython23.py > Log Message: > Just passing -undefined dynamic_lookup isn't enough: we also need to > set > the MACOSX_DEPLOYMENT_TARGET environment variable to 10.3 when calling > the > loader. And we do this with "env" because distutils apparently doesn't > understand environment variable assignments before command names. This is the wrong fix. Distutils is dumber than you think. This patch just breaks C++ compilation in a different way. The correct solution is a patch to distutils of some kind. from distutils/unixccompiler.py:174 if target_lang == "c++" and self.compiler_cxx: linker[0] = self.compiler_cxx[0] self.spawn(linker + ld_args) "linker" is the variable expanded LDSHARED (or whatever comes out of sysconfig.customize_compiler). -bob From Jack.Jansen at cwi.nl Wed Dec 29 11:23:52 2004 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Wed Dec 29 11:23:51 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Mac/OSX fixapplepython23.py, 1.1, 1.2 In-Reply-To: <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com> References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net> <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com> Message-ID: <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl> On 29-dec-04, at 4:35, Bob Ippolito wrote: > This is the wrong fix. Distutils is dumber than you think. This > patch just breaks C++ compilation in a different way. The correct > solution is a patch to distutils of some kind. > > from distutils/unixccompiler.py:174 > > if target_lang == "c++" and self.compiler_cxx: > linker[0] = self.compiler_cxx[0] > self.spawn(linker + ld_args) > > "linker" is the variable expanded LDSHARED (or whatever comes out of > sysconfig.customize_compiler). Bah! Any suggestions as to what to do to get c++ compilation fixed? Also, could you point me to a readily available extension package that uses c++? -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From bob at redivi.com Wed Dec 29 11:40:30 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Dec 29 11:40:38 2004 Subject: [Pythonmac-SIG] Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Mac/OSX fixapplepython23.py, 1.1, 1.2 In-Reply-To: <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl> References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net> <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com> <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl> Message-ID: <0F7FB55E-5986-11D9-9660-000A9567635C@redivi.com> On Dec 29, 2004, at 5:23 AM, Jack Jansen wrote: > On 29-dec-04, at 4:35, Bob Ippolito wrote: > >> This is the wrong fix. Distutils is dumber than you think. This >> patch just breaks C++ compilation in a different way. The correct >> solution is a patch to distutils of some kind. >> >> from distutils/unixccompiler.py:174 >> >> if target_lang == "c++" and self.compiler_cxx: >> linker[0] = self.compiler_cxx[0] >> self.spawn(linker + ld_args) >> >> "linker" is the variable expanded LDSHARED (or whatever comes out of >> sysconfig.customize_compiler). > > Bah! > > Any suggestions as to what to do to get c++ compilation fixed? I can think of two ways: 1. Patch distutils to actually do os.environ['MACOSX_DEPLOYMENT_TARGET'] = '10.3' because CCompiler.spawn doesn't take an environment dict. 2. Patch distutils to skip over environment variables (basically change that "0" into something smarter). This is probably less desirable because who knows where this happens, and who knows what third party compiler subclasses used this original stupid code as a template. In either case, setting the environment variable should only be done if it's not already set, and it should raise if is set to anything lower than 10.3. For example, an existing MACOSX_DEPLOYMENT_TARGET variable of 10.4 should be allowed. Values smaller than 10.3 should not be allowed because they are incompatible with the linker feature we're trying to enable. > Also, could you point me to a readily available extension package that > uses c++? I have no idea, but I remember this issue was brought up at some point during one of the many discussions of this issue. I think it was also mentioned that SciPy's distutils extensions (or is it a fork?) does something similarly stupid when frobbing in a Fortran compiler. -bob From mdehoon at ims.u-tokyo.ac.jp Wed Dec 29 13:13:24 2004 From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon) Date: Wed Dec 29 13:10:17 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Mac/OSX fixapplepython23.py, 1.1, 1.2 In-Reply-To: <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl> References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net> <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com> <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl> Message-ID: <41D29F64.5040308@ims.u-tokyo.ac.jp> Jack Jansen wrote: > Also, could you point me to a readily available extension package that > uses c++? The Bio.Affy and Bio.KDTree extensions in Biopython use c++. See www.biopython.org. --Michiel. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon From rkern at ucsd.edu Thu Dec 30 02:58:22 2004 From: rkern at ucsd.edu (Robert Kern) Date: Thu Dec 30 02:58:36 2004 Subject: [Python-Dev] Re: [Pythonmac-SIG] Re: Re: [Python-checkins] python/dist/src/Mac/OSX fixapplepython23.py, 1.1, 1.2 In-Reply-To: <0F7FB55E-5986-11D9-9660-000A9567635C@redivi.com> References: <E1CjOvO-0002UH-EH@sc8-pr-cvs1.sourceforge.net> <A79A1AF6-594A-11D9-A28B-000A9567635C@redivi.com> <BCDC0AB4-5983-11D9-81BB-000D934FF6B4@cwi.nl> <0F7FB55E-5986-11D9-9660-000A9567635C@redivi.com> Message-ID: <cqvnbt$f3r$1@sea.gmane.org> Bob Ippolito wrote: > I have no idea, but I remember this issue was brought up at some point > during one of the many discussions of this issue. I think it was also > mentioned that SciPy's distutils extensions (or is it a fork?) does > something similarly stupid when frobbing in a Fortran compiler. Extension, and yes it does something similarly stupid. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From martin at v.loewis.de Thu Dec 30 11:46:38 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Dec 30 11:46:38 2004 Subject: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules? In-Reply-To: <20041224123227.GA12925@vicky.ecs.soton.ac.uk> References: <DCF38745-5530-11D9-B212-000A95A8705A@barrys-emacs.org> <41CB50EA.9070100@v.loewis.de> <3E867030-5541-11D9-B212-000A95A8705A@barrys-emacs.org> <20041224123227.GA12925@vicky.ecs.soton.ac.uk> Message-ID: <41D3DC8E.8020100@v.loewis.de> Armin Rigo wrote: > The "extending and embedding" tutorial is similarily out-of-date. This is now fixed in the CVS, both for PC/example_nt, and for Doc/ext/windows.tex. These files still could stand further correction. I forgot to update the output of the examples to 2.4, and the .def file stuff should be removed from the examples, replacing them with the proper macros. Regards, Martin From bac at OCF.Berkeley.EDU Fri Dec 31 09:14:04 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Dec 31 09:14:22 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src setup.py, 1.208, 1.209 In-Reply-To: <41D10550.5000105@ocf.berkeley.edu> References: <E1CflUs-0005gG-6Q@sc8-pr-cvs1.sourceforge.net> <C16E5FE9-5811-11D9-81BB-000D934FF6B4@cwi.nl> <41D10550.5000105@ocf.berkeley.edu> Message-ID: <41D50A4C.4070508@ocf.berkeley.edu> Brett C. wrote: > Jack Jansen wrote: > >> >> On 18-dec-04, at 21:48, bcannon@users.sourceforge.net wrote: >> >>> Update of /cvsroot/python/python/dist/src >>> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21493 >>> >>> Modified Files: >>> setup.py >>> Log Message: >>> Switch from getting LDFLAGS and CPPFLAGS from the environment to the >>> Makefile. >>> This is to avoid a problem that inconsistently comes up where the >>> environment >>> variable is unset while the Makefile clearly has the values set and >>> are used >>> during ``make``. >> >> >> >> Brett, >> I'm building MacPython binary distributions with an LDFLAGS configure >> option, as in >> $PYTHONSRC/configure -C --enable-framework LDFLAGS=-Wl,-x >> >> This has suddenly started failing with a very mysterious error >> message. When the >> make tries to build the extension modules I get >> running build >> running build_ext >> usage: setup.py [options] >> setup.py: error: no such option: -W >> >> Could this somehow be caused by your fix? > > > Yep, it's my doing. I bet optparse is raising an error since it doesn't > have options for -W1 or -x registered with the parser. Anyone off the > top of their head know how to cause optparse to just skip over options > it doesn't know about? > OK, rev. 1.210 has my fix. Now I strip out double dashes (for long options) and any dashes preceding any options that setup.py doesn't care about before passing the string to optparse. That way they just look like interspersed arguments to optparse and it just ignores them. Tested it with your LDFLAGS value and it worked fine for me. > I am on vacation at the moment so I will deal with this when I get back > this weekend. > So I lied and got to it sooner. =) Happy New Year to everyone. -Brett From python at dynkin.com Fri Dec 31 11:33:10 2004 From: python at dynkin.com (George Yoshida) Date: Fri Dec 31 11:33:13 2004 Subject: [Python-Dev] Re: python/dist/src/Doc/ext windows.tex,1.9,1.10 In-Reply-To: <E1CjxnH-0004X8-HB@sc8-pr-cvs1.sourceforge.net> References: <E1CjxnH-0004X8-HB@sc8-pr-cvs1.sourceforge.net> Message-ID: <41D52AE6.2030107@dynkin.com> loewis@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Doc/ext > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17396 > Index: windows.tex > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Doc/ext/windows.tex,v > retrieving revision 1.9 > retrieving revision 1.10 > diff -u -d -r1.9 -r1.10 > --- windows.tex 23 Jan 2004 09:01:56 -0000 1.9 > +++ windows.tex 30 Dec 2004 10:44:32 -0000 1.10 > @@ -142,62 +142,62 @@ [snip] > + Now open the \menuselection{Project \sub spam properties} dialog. > + You only need to change a few settings. Make sure \guilable{All > + Configurations} is selected from the \guilable{Settings for:} Macro names are wrong, which results in a compile error. \guilable should read \guilabel. George