From greg.ewing at canterbury.ac.nz Mon Oct 1 02:44:25 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 01 Oct 2007 13:44:25 +1300 Subject: [Python-ideas] Enhance reload In-Reply-To: <401200.41417.qm@web58907.mail.re1.yahoo.com> References: <401200.41417.qm@web58907.mail.re1.yahoo.com> Message-ID: <470042E9.5090400@canterbury.ac.nz> Joseph Maurer wrote: > Another post I read proposed a Reclass feature that only worked > for classes. You could probably do something similar for functions. Go through the new module, and when you find a function, look to see whether there is something defined in the old module that's also a function. If so, replace the old function's code object with that of the new one. If you extend this to functions inside classes as well, you may even be able to catch bound methods that have been squirrelled away as well. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Mon Oct 1 03:11:38 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 01 Oct 2007 14:11:38 +1300 Subject: [Python-ideas] Enhance reload In-Reply-To: <925784.74407.qm@web58910.mail.re1.yahoo.com> References: <925784.74407.qm@web58910.mail.re1.yahoo.com> Message-ID: <4700494A.80208@canterbury.ac.nz> Joseph Maurer wrote: > Please shot this down this high level implementation if it > won't work in the current code base. Joseph, you're going to have to learn more about how Python works if you want to take this any further. It's not really possible to discuss it in terms as high-level as this. You're making some assumptions about the overall structure that aren't really true. There is an approach that might work, but it's rather different. Instead of trying to load the new code directly into the old module, load it into a new, temporary module and then compare the contents of the two, transferring selected parts where appropriate. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From clarkksv at yahoo.com Mon Oct 1 15:46:02 2007 From: clarkksv at yahoo.com (Joseph Maurer) Date: Mon, 1 Oct 2007 06:46:02 -0700 (PDT) Subject: [Python-ideas] Enhance reload Message-ID: <744635.4702.qm@web58901.mail.re1.yahoo.com> Ok. Thanks for letting me know. Call me Clark. I go by my middle name. I need to change my yahoo account. Does the implementation you describe do the following: 1. Can it be used in per file type of operation (i.e. reload a file of code). It sounds like it can. 2. Will it update variables containing pointers to functions and pointers to methods. As long as the function/method data is being modified in place, this algorithm will work. ----- Original Message ---- From: Greg Ewing To: python-ideas at python.org Sent: Sunday, September 30, 2007 9:11:38 PM Subject: Re: [Python-ideas] Enhance reload Joseph Maurer wrote: > Please shot this down this high level implementation if it > won't work in the current code base. Joseph, you're going to have to learn more about how Python works if you want to take this any further. It's not really possible to discuss it in terms as high-level as this. You're making some assumptions about the overall structure that aren't really true. There is an approach that might work, but it's rather different. Instead of trying to load the new code directly into the old module, load it into a new, temporary module and then compare the contents of the two, transferring selected parts where appropriate. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ _______________________________________________ Python-ideas mailing list Python-ideas at python.org http://mail.python.org/mailman/listinfo/python-ideas ____________________________________________________________________________________ Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz From cmaurer at slickedit.com Mon Oct 1 16:11:39 2007 From: cmaurer at slickedit.com (Clark Maurer) Date: Mon, 1 Oct 2007 10:11:39 -0400 Subject: [Python-ideas] raw strings Message-ID: I'll apologize in advance for this one since I suspect a lot of people have hit this. The current implementation doesn't allow for a trailing backslash in the string. Why don't raw strings in Python work more like C# @"..." strings? Then it would allow for a trailing backslash and you could still get a single quote by two consecutive quotes characters. f=r'c:\src\f' # This is ok and gives you what you want f=r'c:\src\f\' # Compilation error. String is not terminated. f=r'''c:\src\f\''' # This doesn't work either and causes a compilation error. f=r'Here''s another mistake' # This doesn't do what you would think. # You get 'Heres another mistake' f=r'''Here's another mistake''' # This works but being able to use raw strings for this would be nice. f='c:\\src\\f\\' # this works but is ugly I just don't understand the rationale for the current implementation. I thought the intention of raw strings was to allow for backslashes in the string. The current implementation does a bad job at it. Any chance this could be changed with a backward compatibility option? -------------- next part -------------- An HTML attachment was scrubbed... URL: From arno at marooned.org.uk Mon Oct 1 17:45:16 2007 From: arno at marooned.org.uk (Arnaud Delobelle) Date: Mon, 1 Oct 2007 16:45:16 +0100 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: <59D6F476-4C06-4F77-88B1-471E0DB48413@marooned.org.uk> On 1 Oct 2007, at 15:11, Clark Maurer wrote: > I?ll apologize in advance for this one since I suspect a lot of > people have hit this. > > The current implementation doesn?t allow for a trailing backslash > in the string. > > > It's a FAQ: http://www.python.org/doc/faq/general/#why-can-t-raw-strings-r- strings-end-with-a-backslash -- Arnaud From rhamph at gmail.com Mon Oct 1 17:54:43 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 1 Oct 2007 09:54:43 -0600 Subject: [Python-ideas] [Python-Dev] GC Changes In-Reply-To: <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com> <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> Message-ID: [This should be on python-ideas, so I'm replying to there instead of python-dev] On 10/1/07, Justin Tulloss wrote: > Hello, > > I've been doing some tests on removing the GIL, and it's becoming clear that > some basic changes to the garbage collector may be needed in order for this > to happen efficiently. Reference counting as it stands today is not very > scalable. > > I've been looking into a few options, and I'm leaning towards the > implementing IBMs recycler GC ( > http://www.research.ibm.com/people/d/dfb/recycler-publications.html > ) since it is very similar to what is in place now from the users' > perspective. However, I haven't been around the list long enough to really > understand the feeling in the community on GC in the future of the > interpreter. It seems that a full GC might have a lot of benefits in terms > of performance and scalability, and I think that the current gc module is of > the mark-and-sweep variety. Is the trend going to be to move away from > reference counting and towards the mark-and-sweep implementation that > currently exists, or is reference counting a firmly ingrained tradition? Refcounting is fairly firmly ingrained in CPython, but there are conservative GCs for C that mostly work, and other implementations aren't so restricted. The problem with Python is that it produces a *lot* of garbage. Pystones on my box does around a million objects per second and fills up available ram in about 10 seconds. Not only do you need to collect often enough to not fill up the ram, but for *good* performance you need to collect often enough to keep your L1 cache hot. That would seem to demand a generational GC at least. You might as well assume it'll be more expensive than refcounting[1]. The real advantage would be in scalability. Concurrent, parallel GCs are an active field of research though. If you're really interested you should research conservative GCs aimed at C in general, and only minimally interact with CPython (such as to disable the custom allocators.) A good stepping off point is The Memory Management Reference (although it looks like it hasn't been updated in the last few years). If some of my terms are unfamiliar to you, go start reading. ;) http://www.memorymanagement.org/ [1] This statement is only in the context of CPython, of course. There are certainly many situations where a tracing GC performs better. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Mon Oct 1 17:59:54 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 1 Oct 2007 09:59:54 -0600 Subject: [Python-ideas] [Python-Dev] GC Changes In-Reply-To: References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com> <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> Message-ID: On 10/1/07, Adam Olsen wrote: > [This should be on python-ideas, so I'm replying to there instead of python-dev] > > On 10/1/07, Justin Tulloss wrote: > > Hello, > > > > I've been doing some tests on removing the GIL, and it's becoming clear that > > some basic changes to the garbage collector may be needed in order for this > > to happen efficiently. Reference counting as it stands today is not very > > scalable. > > > > I've been looking into a few options, and I'm leaning towards the > > implementing IBMs recycler GC ( > > http://www.research.ibm.com/people/d/dfb/recycler-publications.html > > ) since it is very similar to what is in place now from the users' > > perspective. However, I haven't been around the list long enough to really > > understand the feeling in the community on GC in the future of the > > interpreter. It seems that a full GC might have a lot of benefits in terms > > of performance and scalability, and I think that the current gc module is of > > the mark-and-sweep variety. Is the trend going to be to move away from > > reference counting and towards the mark-and-sweep implementation that > > currently exists, or is reference counting a firmly ingrained tradition? > > Refcounting is fairly firmly ingrained in CPython, but there are > conservative GCs for C that mostly work, and other implementations > aren't so restricted. > > The problem with Python is that it produces a *lot* of garbage. > Pystones on my box does around a million objects per second and fills > up available ram in about 10 seconds. Not only do you need to collect > often enough to not fill up the ram, but for *good* performance you > need to collect often enough to keep your L1 cache hot. That would > seem to demand a generational GC at least. > > You might as well assume it'll be more expensive than refcounting[1]. > The real advantage would be in scalability. Concurrent, parallel GCs > are an active field of research though. If you're really interested > you should research conservative GCs aimed at C in general, and only > minimally interact with CPython (such as to disable the custom > allocators.) Ahh, I forgot a major alternative. You could instead work on an exact GC for PyPy. I personally think it's more interesting to get cooperation from the compiler. ;) > A good stepping off point is The Memory Management Reference (although > it looks like it hasn't been updated in the last few years). If some > of my terms are unfamiliar to you, go start reading. ;) > http://www.memorymanagement.org/ > > > > [1] This statement is only in the context of CPython, of course. > There are certainly many situations where a tracing GC performs > better. > > -- > Adam Olsen, aka Rhamphoryncus > -- Adam Olsen, aka Rhamphoryncus From steven.bethard at gmail.com Mon Oct 1 20:18:11 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 1 Oct 2007 12:18:11 -0600 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: On 10/1/07, Clark Maurer wrote: > The current implementation doesn't allow for a trailing backslash in the > string. I believe that will change in Python 3.0. The discussion is here: http://mail.python.org/pipermail/python-3000/2007-May/007684.html And Ron Adam's current patch is here: http://bugs.python.org/issue1720390 STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From cmaurer at slickedit.com Mon Oct 1 20:30:47 2007 From: cmaurer at slickedit.com (Clark Maurer) Date: Mon, 1 Oct 2007 14:30:47 -0400 Subject: [Python-ideas] raw strings Message-ID: I sure hope it's changed for Python 3.0. The FAQ reason for the current implementation was vague about the processors which were/are the problem. I'm guessing it is an antiquated artifact of limitations in shell line processors. Yuck! -----Original Message----- From: python-ideas-bounces at python.org [mailto:python-ideas-bounces at python.org] On Behalf Of Steven Bethard Sent: Monday, October 01, 2007 2:18 PM To: Python-Ideas Subject: Re: [Python-ideas] raw strings On 10/1/07, Clark Maurer wrote: > The current implementation doesn't allow for a trailing backslash in the > string. I believe that will change in Python 3.0. The discussion is here: http://mail.python.org/pipermail/python-3000/2007-May/007684.html And Ron Adam's current patch is here: http://bugs.python.org/issue1720390 STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy _______________________________________________ Python-ideas mailing list Python-ideas at python.org http://mail.python.org/mailman/listinfo/python-ideas -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Oct 1 20:37:18 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 1 Oct 2007 11:37:18 -0700 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: I'm still against actually. That's why the patch hasn't been applied yet. On 10/1/07, Steven Bethard wrote: > On 10/1/07, Clark Maurer wrote: > > The current implementation doesn't allow for a trailing backslash in the > > string. > > I believe that will change in Python 3.0. > > The discussion is here: > http://mail.python.org/pipermail/python-3000/2007-May/007684.html > > And Ron Adam's current patch is here: > http://bugs.python.org/issue1720390 > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Mon Oct 1 21:05:31 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 01 Oct 2007 21:05:31 +0200 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: Clark Maurer schrieb: [Others have commented about the no-raw-strings-ending-in-backslash thing.] > f=r'Here''s another mistake' # This doesn't do what you would think. > # You get 'Heres another mistake' This is no mistake, and it won't change. The first single quote ends one string literal, and the second starts another, and they are concatenated in the parsing stage. > f=r'''Here's another mistake''' # This works but being able to use raw > # strings for this would be nice. Actually, this is a raw string too. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From steven.bethard at gmail.com Mon Oct 1 21:42:27 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 1 Oct 2007 13:42:27 -0600 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: > On 10/1/07, Steven Bethard wrote: > > On 10/1/07, Clark Maurer wrote: > > > The current implementation doesn't allow for a trailing backslash in the > > > string. > > > > I believe that will change in Python 3.0. > > > > The discussion is here: > > http://mail.python.org/pipermail/python-3000/2007-May/007684.html > > > > And Ron Adam's current patch is here: > > http://bugs.python.org/issue1720390 On 10/1/07, Guido van Rossum wrote: > I'm still against actually. That's why the patch hasn't been applied yet. Sorry, my mistake. I read the thread as being somewhat in support of the change. Anyway, to the OP, if you want to make this happen, you should help Ron out with his patch. (Code has a much better chance of convincing Guido than anything else does.) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Mon Oct 1 21:53:41 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 1 Oct 2007 12:53:41 -0700 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: On 10/1/07, Steven Bethard wrote: > > On 10/1/07, Steven Bethard wrote: > > > On 10/1/07, Clark Maurer wrote: > > > > The current implementation doesn't allow for a trailing backslash in the > > > > string. > > > > > > I believe that will change in Python 3.0. > > > > > > The discussion is here: > > > http://mail.python.org/pipermail/python-3000/2007-May/007684.html > > > > > > And Ron Adam's current patch is here: > > > http://bugs.python.org/issue1720390 > > On 10/1/07, Guido van Rossum wrote: > > I'm still against actually. That's why the patch hasn't been applied yet. > > Sorry, my mistake. I read the thread as being somewhat in support of the change. I admit I've been wobbling a lot on this. > Anyway, to the OP, if you want to make this happen, you should help > Ron out with his patch. (Code has a much better chance of convincing > Guido than anything else does.) Not in this case. It's more the philosophical distinction -- are raw strings meant primarily to hold regexes or Windows pathnames? These two use cases have opposite requirements for trailing backslash treatment. I know the original use case that caused them to be added to the language is regexes, and that's still the only one I use on a regular basis. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From phd at phd.pp.ru Mon Oct 1 22:01:43 2007 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 2 Oct 2007 00:01:43 +0400 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: <20071001200143.GA27547@phd.pp.ru> On Mon, Oct 01, 2007 at 12:53:41PM -0700, Guido van Rossum wrote: > Not in this case. It's more the philosophical distinction -- are raw > strings meant primarily to hold regexes or Windows pathnames? These > two use cases have opposite requirements for trailing backslash > treatment. I know the original use case that caused them to be added > to the language is regexes, and that's still the only one I use on a > regular basis. Then just rename them to "regex-strings"; prefix 'r' is stll ok. ;) Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From steven.bethard at gmail.com Mon Oct 1 22:23:48 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 1 Oct 2007 14:23:48 -0600 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: On 10/1/07, Clark Maurer wrote: > The current implementation doesn't allow for a trailing backslash in the > string. On 10/1/07, Guido van Rossum wrote: > It's more the philosophical distinction -- are raw > strings meant primarily to hold regexes or Windows pathnames? These > two use cases have opposite requirements for trailing backslash > treatment. I know the original use case that caused them to be added > to the language is regexes, and that's still the only one I use on a > regular basis. FWLIW, I'm in exactly the opposite boat. I use very few regular expressions that include quotes, and when I do run into one I always use the appropriate quote type so that I don't need to escape the quotes. On the other hand, I write quite a few Windows paths, and probably make the final-backslash mistake once a week. (It's quickly flagged and fixed, but I still make it.) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Mon Oct 1 22:39:46 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 1 Oct 2007 13:39:46 -0700 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: (Clark: I don't want to discuss this offline. On the list it goes.) Quote doubling isn't a viable option for Python -- I don't believe it's sane to have both backslashes and quote-double as escape mechanisms. Of course in C# the trailing \ is the main use case -- after all it's a Microsoft product. While for some Windows users this may be a nuisance, I don't think they are in the majority amongst Python users. --Guido On 10/1/07, Clark Maurer wrote: > I'll do what I can to sway you. > > Guido, please....pretty please with sugar on top (hehehe). I've been > spoiled with languages which do better raw string. I designed Slick-C > which I've been using for years and it does not have this problem. I > always type Windows paths using the Slick-C equivalent of raw strings. > My implementation came from REXX (I'm no spring chicken- I think you've > heard of it too :-). Now that C# has gone with this implementation, that > pretty much makes it the modern way to do this. C# has a lot more clout > than Slick-C. Yes, these style strings are intended for regexes AND > Windows paths. Just think, you can get rid of one FAQ. It's easier to > document. The down side is that there will be some backward > compatibility issues. I will admit, compared to other issues this is a > small one. > > Trailing backslash isn't the only problem but it's the bigger one. Two > quotes should be one single quote. Otherwise, specifying both quote > characters in regexes is an issue. Yes, I've done this in regular > expressions. This change could cause some backward compatibility > problems as well. > > Clark > -----Original Message----- > From: python-ideas-bounces at python.org > [mailto:python-ideas-bounces at python.org] On Behalf Of Guido van Rossum > Sent: Monday, October 01, 2007 3:54 PM > To: Steven Bethard > Cc: Python-Ideas > Subject: Re: [Python-ideas] raw strings > > On 10/1/07, Steven Bethard wrote: > > > On 10/1/07, Steven Bethard wrote: > > > > On 10/1/07, Clark Maurer wrote: > > > > > The current implementation doesn't allow for a trailing > backslash in the > > > > > string. > > > > > > > > I believe that will change in Python 3.0. > > > > > > > > The discussion is here: > > > > http://mail.python.org/pipermail/python-3000/2007-May/007684.html > > > > > > > > And Ron Adam's current patch is here: > > > > http://bugs.python.org/issue1720390 > > > > On 10/1/07, Guido van Rossum wrote: > > > I'm still against actually. That's why the patch hasn't been applied > yet. > > > > Sorry, my mistake. I read the thread as being somewhat in support of > the change. > > I admit I've been wobbling a lot on this. > > > Anyway, to the OP, if you want to make this happen, you should help > > Ron out with his patch. (Code has a much better chance of convincing > > Guido than anything else does.) > > Not in this case. It's more the philosophical distinction -- are raw > strings meant primarily to hold regexes or Windows pathnames? These > two use cases have opposite requirements for trailing backslash > treatment. I know the original use case that caused them to be added > to the language is regexes, and that's still the only one I use on a > regular basis. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From taleinat at gmail.com Mon Oct 1 22:51:26 2007 From: taleinat at gmail.com (Tal Einat) Date: Mon, 1 Oct 2007 22:51:26 +0200 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: <7afdee2f0710011351n129662d5la05269daae645612@mail.gmail.com> On 10/1/07, Guido van Rossum wrote: > Not in this case. It's more the philosophical distinction -- are raw > strings meant primarily to hold regexes or Windows pathnames? These > two use cases have opposite requirements for trailing backslash > treatment. I know the original use case that caused them to be added > to the language is regexes, and that's still the only one I use on a > regular basis. > >From a teaching (and simplicity) viewpoint, having these be "raw strings" instead of "regexp strings" would be better. The current behavior is a snag I always have to mention when teaching Python, and students are often caught by this once or twice. As for my experience with regexps, the current behavior is only useful when using both single and double quotes in a single regexp string; I can't recall when I last did so. On 10/1/07, Clark Maurer wrote: > Two quotes should be one single quote. Otherwise, specifying both > quote characters in regexes is an issue. >From the teaching point of view, the preferred behavior for "raw strings" would be no escaping for quotes - make them escape-less, period. Mixing single and double quotes would be done by concatenating strings. With Python's terse string concatenation I don't see the need for special escaping for only one character to support a rare use case. - Tal From steven.bethard at gmail.com Mon Oct 1 22:56:45 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 1 Oct 2007 14:56:45 -0600 Subject: [Python-ideas] raw strings In-Reply-To: <7afdee2f0710011351n129662d5la05269daae645612@mail.gmail.com> References: <7afdee2f0710011351n129662d5la05269daae645612@mail.gmail.com> Message-ID: On 10/1/07, Tal Einat wrote: > On 10/1/07, Guido van Rossum wrote: > > Not in this case. It's more the philosophical distinction -- are raw > > strings meant primarily to hold regexes or Windows pathnames? These > > two use cases have opposite requirements for trailing backslash > > treatment. I know the original use case that caused them to be added > > to the language is regexes, and that's still the only one I use on a > > regular basis. > > > > >From a teaching (and simplicity) viewpoint, having these be "raw > strings" instead of "regexp strings" would be better. The current > behavior is a snag I always have to mention when teaching Python, and > students are often caught by this once or twice. > > As for my experience with regexps, the current behavior is only useful > when using both single and double quotes in a single regexp string; I > can't recall when I last did so. I think this is the crux of the matter. It's not really about comparing use of regular expressions with use of Windows paths. It's about comparing use of regular expressions *that include both types of quotes* and use of Windows paths. Of course, I don't really know how to perform this sort of comparison on a large scale either. ;-) > From the teaching point of view, the preferred behavior for "raw > strings" would be no escaping for quotes - make them escape-less, > period. Exactly my feeling. Thanks for putting it so clearly. =) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From g.brandl at gmx.net Mon Oct 1 23:03:07 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 01 Oct 2007 23:03:07 +0200 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > (Clark: I don't want to discuss this offline. On the list it goes.) > > Quote doubling isn't a viable option for Python -- I don't believe > it's sane to have both backslashes and quote-double as escape > mechanisms. > > Of course in C# the trailing \ is the main use case -- after all it's > a Microsoft product. > > While for some Windows users this may be a nuisance, I don't think > they are in the majority amongst Python users. But why upset them? What is the reason for not allowing trailing backslashes in raw strings, except more code in the tokenizer? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From cmaurer at slickedit.com Mon Oct 1 23:13:23 2007 From: cmaurer at slickedit.com (Clark Maurer) Date: Mon, 1 Oct 2007 17:13:23 -0400 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: Guido, I'm not proposing both escape mechanisms. Quote-doubling is a mechanism which should be used ONLY for raw strings. Let be very specific how this is done: r"Here's a string with a single quote" r'"Heres a double quoted string"' r'["'']' # This regex has both quotes in it r"[""']" # This regex has both quotes in it Quote doubling is only for the quote you started the raw string with. This would be consistent with C#, REXX, and Slick-C. It sounds like you've never used a language with this construct. It's very natural to me. This would make the raw strings implementation complete. Clark -----Original Message----- From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum Sent: Monday, October 01, 2007 4:40 PM To: Clark Maurer Cc: Python-Ideas Subject: Re: [Python-ideas] raw strings (Clark: I don't want to discuss this offline. On the list it goes.) Quote doubling isn't a viable option for Python -- I don't believe it's sane to have both backslashes and quote-double as escape mechanisms. Of course in C# the trailing \ is the main use case -- after all it's a Microsoft product. While for some Windows users this may be a nuisance, I don't think they are in the majority amongst Python users. --Guido On 10/1/07, Clark Maurer wrote: > I'll do what I can to sway you. > > Guido, please....pretty please with sugar on top (hehehe). I've been > spoiled with languages which do better raw string. I designed Slick-C > which I've been using for years and it does not have this problem. I > always type Windows paths using the Slick-C equivalent of raw strings. > My implementation came from REXX (I'm no spring chicken- I think you've > heard of it too :-). Now that C# has gone with this implementation, that > pretty much makes it the modern way to do this. C# has a lot more clout > than Slick-C. Yes, these style strings are intended for regexes AND > Windows paths. Just think, you can get rid of one FAQ. It's easier to > document. The down side is that there will be some backward > compatibility issues. I will admit, compared to other issues this is a > small one. > > Trailing backslash isn't the only problem but it's the bigger one. Two > quotes should be one single quote. Otherwise, specifying both quote > characters in regexes is an issue. Yes, I've done this in regular > expressions. This change could cause some backward compatibility > problems as well. > > Clark > -----Original Message----- > From: python-ideas-bounces at python.org > [mailto:python-ideas-bounces at python.org] On Behalf Of Guido van Rossum > Sent: Monday, October 01, 2007 3:54 PM > To: Steven Bethard > Cc: Python-Ideas > Subject: Re: [Python-ideas] raw strings > > On 10/1/07, Steven Bethard wrote: > > > On 10/1/07, Steven Bethard wrote: > > > > On 10/1/07, Clark Maurer wrote: > > > > > The current implementation doesn't allow for a trailing > backslash in the > > > > > string. > > > > > > > > I believe that will change in Python 3.0. > > > > > > > > The discussion is here: > > > > http://mail.python.org/pipermail/python-3000/2007-May/007684.html > > > > > > > > And Ron Adam's current patch is here: > > > > http://bugs.python.org/issue1720390 > > > > On 10/1/07, Guido van Rossum wrote: > > > I'm still against actually. That's why the patch hasn't been applied > yet. > > > > Sorry, my mistake. I read the thread as being somewhat in support of > the change. > > I admit I've been wobbling a lot on this. > > > Anyway, to the OP, if you want to make this happen, you should help > > Ron out with his patch. (Code has a much better chance of convincing > > Guido than anything else does.) > > Not in this case. It's more the philosophical distinction -- are raw > strings meant primarily to hold regexes or Windows pathnames? These > two use cases have opposite requirements for trailing backslash > treatment. I know the original use case that caused them to be added > to the language is regexes, and that's still the only one I use on a > regular basis. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Mon Oct 1 23:22:33 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 1 Oct 2007 17:22:33 -0400 Subject: [Python-ideas] raw strings In-Reply-To: References: <7afdee2f0710011351n129662d5la05269daae645612@mail.gmail.com> Message-ID: On 10/1/07, Steven Bethard wrote: > On 10/1/07, Tal Einat wrote: > > As for my experience with regexps, the current behavior is only useful > > when using both single and double quotes in a single regexp string; I > > can't recall when I last did so. I have seen regular expressions that look for a double-quoted string which might contain an apostrophe, but don't remember writing one. Other than that, I can't remember *ever* needing to unless I was really (at some level) saying "either of these two"; I also can't remember doing it without being annoyed at having to figure out how many ticks were there, and whether it was double-single or three singles, or what. For me, usability and readability would both improve by offering a named special character, such as \q. > > From the teaching point of view, the preferred behavior for "raw > > strings" would be no escaping for quotes - make them escape-less, > > period. > Exactly my feeling. Thanks for putting it so clearly. =) Agreed. And also from a "what do *I* have to remember", or a "how does this interact when it gets passed to something else" perspective, because it would remove one level of escaping. -jJ From guido at python.org Mon Oct 1 23:27:02 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 1 Oct 2007 14:27:02 -0700 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: On 10/1/07, Georg Brandl wrote: > Guido van Rossum schrieb: > > (Clark: I don't want to discuss this offline. On the list it goes.) > > > > Quote doubling isn't a viable option for Python -- I don't believe > > it's sane to have both backslashes and quote-double as escape > > mechanisms. > > > > Of course in C# the trailing \ is the main use case -- after all it's > > a Microsoft product. > > > > While for some Windows users this may be a nuisance, I don't think > > they are in the majority amongst Python users. > > But why upset them? What is the reason for not allowing trailing > backslashes in raw strings, except more code in the tokenizer? Well, everybody else who is tokenizing Python will also have to have more code. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Mon Oct 1 23:27:15 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 1 Oct 2007 17:27:15 -0400 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: On 10/1/07, Clark Maurer wrote: > Quote doubling is only for the quote you started the raw string with. Understood. > This would be consistent with C#, REXX, and Slick-C. It sounds like > you've never used a language with this construct. It's very natural to > me. Even python does this, with %% on format strings -- but it is only natural to someone who already has some experience programming. -jJ From guido at python.org Mon Oct 1 23:30:02 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 1 Oct 2007 14:30:02 -0700 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: On 10/1/07, Clark Maurer wrote: > Guido, I'm not proposing both escape mechanisms. Quote-doubling is a > mechanism which should be used ONLY for raw strings. Let be very > specific how this is done: > > r"Here's a string with a single quote" > r'"Heres a double quoted string"' > r'["'']' # This regex has both quotes in it > r"[""']" # This regex has both quotes in it > > Quote doubling is only for the quote you started the raw string with. > This would be consistent with C#, REXX, and Slick-C. It sounds like > you've never used a language with this construct. It's very natural to > me. This would make the raw strings implementation complete. As long as we're making personal comments, who cares about REXX or Slick-C? How many users do they have? And yes, I've used this for years in Pascal, so I'm well familiar with it. The whole problem is that when using regexes you *do* have to deal with both escape mechanisms (one for the string literal tokenization and one for the re module). And that's unacceptable for me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From cmaurer at slickedit.com Tue Oct 2 00:06:13 2007 From: cmaurer at slickedit.com (Clark Maurer) Date: Mon, 1 Oct 2007 18:06:13 -0400 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: It sounds like nobody likes quote-doubling. Supporting a trailing backslash would alleviate the need for an FAQ item and the typical confusion. Sounds like most agree on that point. Sorry about sounding personal. -----Original Message----- From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum Sent: Monday, October 01, 2007 5:30 PM To: Clark Maurer Cc: python-ideas at python.org Subject: Re: [Python-ideas] raw strings On 10/1/07, Clark Maurer wrote: > Guido, I'm not proposing both escape mechanisms. Quote-doubling is a > mechanism which should be used ONLY for raw strings. Let be very > specific how this is done: > > r"Here's a string with a single quote" > r'"Heres a double quoted string"' > r'["'']' # This regex has both quotes in it > r"[""']" # This regex has both quotes in it > > Quote doubling is only for the quote you started the raw string with. > This would be consistent with C#, REXX, and Slick-C. It sounds like > you've never used a language with this construct. It's very natural to > me. This would make the raw strings implementation complete. As long as we're making personal comments, who cares about REXX or Slick-C? How many users do they have? And yes, I've used this for years in Pascal, so I'm well familiar with it. The whole problem is that when using regexes you *do* have to deal with both escape mechanisms (one for the string literal tokenization and one for the re module). And that's unacceptable for me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From santagada at gmail.com Tue Oct 2 00:32:36 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Mon, 1 Oct 2007 19:32:36 -0300 Subject: [Python-ideas] [Python-Dev] GC Changes In-Reply-To: References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com> <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> Message-ID: Em Oct 1, 2007, ?s 12:59 PM, Adam Olsen escreveu: > Ahh, I forgot a major alternative. You could instead work on an exact > GC for PyPy. I personally think it's more interesting to get > cooperation from the compiler. ;) > That would be great, they are looking for people wanting to do that. It is hard work, but if you think you can do that for CPython you can do it on PyPy. Please contact the people using http://codespeak.net/ pypy/dist/pypy/doc/contact.html. Usually the IRC channel (#pypy on freenode.net) is the fastest way to get started. -- Leonardo Santagada From greg.ewing at canterbury.ac.nz Tue Oct 2 02:54:32 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 02 Oct 2007 13:54:32 +1300 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: <470196C8.60305@canterbury.ac.nz> Clark Maurer wrote: > I just don?t understand the rationale for the current implementation. > I thought the intention of raw strings was to allow for backslashes in > the string. That's not *exactly* true. The intention is to allow backslashes that are *being used to escape something*, but are to be interpreted by whatever is using the string, not by Python itself. The original use case was regular expressions, I believe. In that case, there is no need to be able to put a backslash at the end of a string. Your suggestion of doubling quotes would actually interfere with this use case, because currently you can write r"a\"b" and it will be correctly interpreted as a regular expression with an escaped quote in it. Under your scheme, it would have to be written r"a\""b" and then it would no longer be WYSIWYG. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Oct 2 02:58:12 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 02 Oct 2007 13:58:12 +1300 Subject: [Python-ideas] [Python-Dev] GC Changes In-Reply-To: References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com> <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> Message-ID: <470197A4.5080205@canterbury.ac.nz> Adam Olsen wrote: > Refcounting is fairly firmly ingrained in CPython, but there are > conservative GCs for C that mostly work, Unfortunately, "mostly works" isn't good enough for something to be incorporated into CPython. It needs to work completely reliably and very portably. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Oct 2 03:09:17 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 02 Oct 2007 14:09:17 +1300 Subject: [Python-ideas] raw strings In-Reply-To: <20071001200143.GA27547@phd.pp.ru> References: <20071001200143.GA27547@phd.pp.ru> Message-ID: <47019A3D.8020408@canterbury.ac.nz> Oleg Broytmann wrote: > Then just rename them to "regex-strings"; prefix 'r' is stll ok. ;) But then all the Perl-heads will complain that they don't automatically get compiled into regex objects... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Oct 2 03:11:28 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 02 Oct 2007 14:11:28 +1300 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: <47019AC0.8010707@canterbury.ac.nz> Steven Bethard wrote: > I write quite a few Windows paths, and > probably make the final-backslash mistake once a week. If you were using os.path.join(), as you should be, you wouldn't ever have to write a trailing backslash in a path in the first place. Or any backslashes at all, for that matter. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From rhamph at gmail.com Tue Oct 2 03:07:47 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 1 Oct 2007 19:07:47 -0600 Subject: [Python-ideas] [Python-Dev] GC Changes In-Reply-To: <470197A4.5080205@canterbury.ac.nz> References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com> <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> <470197A4.5080205@canterbury.ac.nz> Message-ID: On 10/1/07, Greg Ewing wrote: > Adam Olsen wrote: > > Refcounting is fairly firmly ingrained in CPython, but there are > > conservative GCs for C that mostly work, > > Unfortunately, "mostly works" isn't good enough for > something to be incorporated into CPython. It needs to > work completely reliably and very portably. Indeed. An example of how they can fail would be python linking the boehm GC (which overrides malloc and free), then dynamically loading python from another app. By the time boehm overrides malloc and free they've already been used significantly. Another example is any sort of custom allocator. At best the conservative GC will see their heap as a single giant object (and be unable to release objects within it). At worst (if they allocate the heap via mmap rather than the overridden malloc) it won't even know the heap exists, not include it in the root set, and prematurely free objects it references. -- Adam Olsen, aka Rhamphoryncus From greg.ewing at canterbury.ac.nz Tue Oct 2 03:16:37 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 02 Oct 2007 14:16:37 +1300 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: <47019BF5.4000800@canterbury.ac.nz> Clark Maurer wrote: > Quote-doubling is a > mechanism which should be used ONLY for raw strings. But there would be two quoting mechanisms in the *language*, and you would have to remember which type of string used which. > This would be consistent with C#, REXX, and Slick-C. > It's very natural to me. But *un*-natural to anyone not familiar with those languages, and confusing to anyone expecting consistency between one kind of string and another within the same language. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From aahz at pythoncraft.com Tue Oct 2 04:47:45 2007 From: aahz at pythoncraft.com (Aahz) Date: Mon, 1 Oct 2007 19:47:45 -0700 Subject: [Python-ideas] [Python-Dev] GC Changes In-Reply-To: <470192D7.5020504@canterbury.ac.nz> References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com> <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> <470192D7.5020504@canterbury.ac.nz> Message-ID: <20071002024745.GA21172@panix.com> [xposted to python-ideas, reply-to python-ideas, leaving python-dev in to correct misinformation] On Tue, Oct 02, 2007, Greg Ewing wrote: > > The cyclic GC kicks in when memory is running low. Not at all. The sole and only basis for GC is number of allocations compared to number of de-allocations. See http://docs.python.org/lib/module-gc.html -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The best way to get information on Usenet is not to ask a question, but to post the wrong information. From steven.bethard at gmail.com Tue Oct 2 06:28:14 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 1 Oct 2007 22:28:14 -0600 Subject: [Python-ideas] raw strings In-Reply-To: <47019AC0.8010707@canterbury.ac.nz> References: <47019AC0.8010707@canterbury.ac.nz> Message-ID: On 10/1/07, Greg Ewing wrote: > Steven Bethard wrote: > > I write quite a few Windows paths, and > > probably make the final-backslash mistake once a week. > > If you were using os.path.join(), as you should be, > you wouldn't ever have to write a trailing backslash > in a path in the first place. Or any backslashes at > all, for that matter. Sure, but why would I waste my time with that at the interactive prompt? I'm trying to get something done quickly. I can copy-paste from just about anywhere something like this:: C:\Documents and Settings\...\My Documents\projects\causals\treebank-verb-conj-801-1100.xml If I have to use os.path.join, it'll take me 10 times as long to input it. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From rrr at ronadam.com Tue Oct 2 06:44:29 2007 From: rrr at ronadam.com (Ron Adam) Date: Mon, 01 Oct 2007 23:44:29 -0500 Subject: [Python-ideas] raw strings In-Reply-To: References: Message-ID: <4701CCAD.6030008@ronadam.com> Guido van Rossum wrote: > On 10/1/07, Clark Maurer wrote: >> Guido, I'm not proposing both escape mechanisms. Quote-doubling is a >> mechanism which should be used ONLY for raw strings. Let be very >> specific how this is done: >> >> r"Here's a string with a single quote" >> r'"Heres a double quoted string"' >> r'["'']' # This regex has both quotes in it >> r"[""']" # This regex has both quotes in it >> >> Quote doubling is only for the quote you started the raw string with. >> This would be consistent with C#, REXX, and Slick-C. It sounds like >> you've never used a language with this construct. It's very natural to >> me. This would make the raw strings implementation complete. > > As long as we're making personal comments, who cares about REXX or > Slick-C? How many users do they have? And yes, I've used this for > years in Pascal, so I'm well familiar with it. The whole problem is > that when using regexes you *do* have to deal with both escape > mechanisms (one for the string literal tokenization and one for the re > module). And that's unacceptable for me. With pythons current 4 different ways to quote, I don't think quote doubling is needed. Yes, it's the duel escape mechanisms that makes regexes more complex than it needs to be. Ron From talin at acm.org Tue Oct 2 08:06:46 2007 From: talin at acm.org (Talin) Date: Mon, 01 Oct 2007 23:06:46 -0700 Subject: [Python-ideas] [Python-Dev] GC Changes In-Reply-To: References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com> <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com> Message-ID: <4701DFF6.20708@acm.org> Adam Olsen wrote: > Ahh, I forgot a major alternative. You could instead work on an exact > GC for PyPy. I personally think it's more interesting to get > cooperation from the compiler. ;) The open source world sorely needs an exact garbage collector, which is why I am in the process of writing one. Right now its going very slowly, as I've been able to only commit a few hours a week to the work. One difficulty in making a FOSS garbage collector is that exact collectors tend to be tightly coupled to the specific object model of the runtime environment. The design of the collector puts a lot of constraints on the design of the object model, and vice versa. This makes it difficult to create a collector that works with a lot of different languages. As you already know, PyPy is currently targeting LLVM as a backend. LLVM provides an abstract interface for garbage collection that solves some of these problems. Because the collector interface is defined at the virtual instruction level, before optimization takes place, it means that some of the overhead that would be incurred by making a completely generic callback interface can be avoided - that is, the abstraction levels needed for a loosely-coupled collector can be optimized away by LLVM's optimizer. Based on a study of the LLVM code and docs, I have come to the belief that it would be possible to create a generic collector implementation which would work with a fairly broad class of both object models and collection algorithms. In other words, while it might not support every possible language out there, it would be possible to support a fairly wide subset. So far, what I've got is a general purpose heap implementation, which is loosely inspired by (although not copied from) the popular dlmalloc implementation and a few others, and also incorporates a number of features which would be needed by a collector algorithm (such as reserved space for collector state bits and type tags in the object allocation header.) I've also got some utility classes that would be useful to a collector, such as a lock-free implementation of work queues. I don't have an actual collector working yet, and I'm not going to post a link to the code here because it's not ready for public viewing yet. In any case, if anyone is interested in discussing this further, feel free to email me privately. Since this isn't directly related to CPython or PyPy, I don't want to clutter up the discussion lists here with issues related to garbage collection. -- Talin From aahz at pythoncraft.com Tue Oct 2 15:07:57 2007 From: aahz at pythoncraft.com (Aahz) Date: Tue, 2 Oct 2007 06:07:57 -0700 Subject: [Python-ideas] os.path.join() failure In-Reply-To: <47019AC0.8010707@canterbury.ac.nz> References: <47019AC0.8010707@canterbury.ac.nz> Message-ID: <20071002130757.GA4307@panix.com> On Tue, Oct 02, 2007, Greg Ewing wrote: > Steven Bethard wrote: >> >> I write quite a few Windows paths, and probably make the >> final-backslash mistake once a week. > > If you were using os.path.join(), as you should be, you wouldn't ever > have to write a trailing backslash in a path in the first place. Or > any backslashes at all, for that matter. Wrong. And I just got bitten by this yesterday (no backslashes at all). Consider the difference between cp -a foo bar and cp -a foo/ bar The two are almost identical *except* when foo is a symlink to a directory, then the first form copies the symlink instead of giving a brand-new directory. Which, since foo/ is a template dir that gets modified after copying was a very bad thing. (We moved servers around over the weekend and for convenience's sake made foo a symlink to a new location.) And yes, we *were* using os.path.join(). So trailing slashes (or backslashes) are in fact sometimes required. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The best way to get information on Usenet is not to ask a question, but to post the wrong information. From jh at improva.dk Tue Oct 2 15:36:53 2007 From: jh at improva.dk (Jacob Holm) Date: Tue, 02 Oct 2007 15:36:53 +0200 Subject: [Python-ideas] os.path.join() failure In-Reply-To: <20071002130757.GA4307@panix.com> References: <47019AC0.8010707@canterbury.ac.nz> <20071002130757.GA4307@panix.com> Message-ID: <47024975.6020309@improva.dk> Aahz wrote: > On Tue, Oct 02, 2007, Greg Ewing wrote: > >> Steven Bethard wrote: >> >>> I write quite a few Windows paths, and probably make the >>> final-backslash mistake once a week. >>> >> If you were using os.path.join(), as you should be, you wouldn't ever >> have to write a trailing backslash in a path in the first place. Or >> any backslashes at all, for that matter. >> > > Wrong. And I just got bitten by this yesterday (no backslashes at all). > Consider the difference between > > cp -a foo bar > > and > > cp -a foo/ bar > > The two are almost identical *except* when foo is a symlink to a > directory, then the first form copies the symlink instead of giving a > brand-new directory. Which, since foo/ is a template dir that gets > modified after copying was a very bad thing. (We moved servers around > over the weekend and for convenience's sake made foo a symlink to a new > location.) > > And yes, we *were* using os.path.join(). So trailing slashes (or > backslashes) are in fact sometimes required. > And that is easily doable using os.path.join. Just use an empty string as the last argument. >>> import os.path >>> os.path.join('foo','') foo/ So no, you do *not* need to write trailing slashes. -- Jacob Holm CTO Improva ApS From greg.ewing at canterbury.ac.nz Wed Oct 3 01:18:11 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 03 Oct 2007 11:18:11 +1200 Subject: [Python-ideas] os.path.join() failure In-Reply-To: <20071002130757.GA4307@panix.com> References: <47019AC0.8010707@canterbury.ac.nz> <20071002130757.GA4307@panix.com> Message-ID: <4702D1B3.7030002@canterbury.ac.nz> Aahz wrote: > cp -a foo bar > > and > > cp -a foo/ bar > > The two are almost identical *except* when foo is a symlink to a > directory, Hm. I would call this a misfeature of the version of cp you're using. In Unix, a trailing slash is *never* supposed to be significant. -- Greg From aahz at pythoncraft.com Wed Oct 3 02:05:04 2007 From: aahz at pythoncraft.com (Aahz) Date: Tue, 2 Oct 2007 17:05:04 -0700 Subject: [Python-ideas] os.path.join() failure In-Reply-To: <4702D1B3.7030002@canterbury.ac.nz> References: <47019AC0.8010707@canterbury.ac.nz> <20071002130757.GA4307@panix.com> <4702D1B3.7030002@canterbury.ac.nz> Message-ID: <20071003000504.GA14812@panix.com> On Wed, Oct 03, 2007, Greg Ewing wrote: > Aahz wrote: >> >> cp -a foo bar >> >> and >> >> cp -a foo/ bar >> >> The two are almost identical *except* when foo is a symlink to a >> directory, > > Hm. I would call this a misfeature of the version of cp you're using. > > In Unix, a trailing slash is *never* supposed to be significant. Well, it often is. See e.g. man rsync. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The best way to get information on Usenet is not to ask a question, but to post the wrong information. From adam at atlas.st Mon Oct 8 05:33:51 2007 From: adam at atlas.st (Adam Atlas) Date: Sun, 7 Oct 2007 23:33:51 -0400 Subject: [Python-ideas] An easier syntax for writing decorators (& similar things)? Message-ID: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> When writing decorators especially when it's one that needs arguments other than the function to be wrapped, it often gets rather ugly... def dec(a, b, foo=bar): def inner(func): def something(*a, **k): ...stuff... return func(*a, **k) return something return inner Perhaps we could allow functions to be defined with multiple argument lists, basically partially applying the function until all of them are filled. (Sort of like currying, but sort of not.) def dec(a, b, foo=bar)(func)(*a, **k): ...stuff... return func(*a, **k) So, calling `dec` will fill the first argument list and return a callable, which when called will fill the second argument list and return a third callable, which will be the fully-decorated function. Basically, exactly as it looks -- def func(a)(b)(c) is called as func (1)(2)(3). Except, obviously, you can partially apply it by only calling the first one or two or however many. I'm not sure how this would look internally, but I imagine each successive call would return an object something like a partial. I expect that the main argument against this will be that it is not a common enough idiom to warrant adding syntax. Perhaps; I don't know. The decorator pattern is very useful (and not only in the @blah function decorator sense -- also the future class decorators, WSGI middleware, etc.), and I do think it makes their definitions quite a bit nicer and easier to read. Any thoughts? From brett at python.org Mon Oct 8 05:49:45 2007 From: brett at python.org (Brett Cannon) Date: Sun, 7 Oct 2007 20:49:45 -0700 Subject: [Python-ideas] An easier syntax for writing decorators (& similar things)? In-Reply-To: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> References: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> Message-ID: On 10/7/07, Adam Atlas wrote: > When writing decorators especially when it's one that needs arguments > other than the function to be wrapped, it often gets rather ugly... > > def dec(a, b, foo=bar): > def inner(func): > def something(*a, **k): > ...stuff... > return func(*a, **k) > return something > return inner > > Perhaps we could allow functions to be defined with multiple argument > lists, basically partially applying the function until all of them > are filled. (Sort of like currying, but sort of not.) > > def dec(a, b, foo=bar)(func)(*a, **k): > ...stuff... > return func(*a, **k) > > So, calling `dec` will fill the first argument list and return a > callable, which when called will fill the second argument list and > return a third callable, which will be the fully-decorated function. > Basically, exactly as it looks -- def func(a)(b)(c) is called as func > (1)(2)(3). Except, obviously, you can partially apply it by only > calling the first one or two or however many. I'm not sure how this > would look internally, but I imagine each successive call would > return an object something like a partial. > > I expect that the main argument against this will be that it is not a > common enough idiom to warrant adding syntax. Perhaps; I don't know. > The decorator pattern is very useful (and not only in the @blah > function decorator sense -- also the future class decorators, WSGI > middleware, etc.), and I do think it makes their definitions quite a > bit nicer and easier to read. Any thoughts? My argument against it is it's ugly and hard to notice. It is not obvious unless you really look carefully at the signature to notice that there are three sets of parentheses there. -Brett From arno at marooned.org.uk Mon Oct 8 11:57:12 2007 From: arno at marooned.org.uk (Arnaud Delobelle) Date: Mon, 8 Oct 2007 10:57:12 +0100 (BST) Subject: [Python-ideas] An easier syntax for writing decorators (& similar things)? In-Reply-To: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> References: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> Message-ID: <35048.194.154.22.39.1191837432.squirrel@www.marooned.org.uk> On Mon, October 8, 2007 4:33 am, Adam Atlas wrote: > When writing decorators especially when it's one that needs arguments > other than the function to be wrapped, it often gets rather ugly... > > def dec(a, b, foo=bar): > def inner(func): > def something(*a, **k): > ...stuff... > return func(*a, **k) > return something > return inner > > Perhaps we could allow functions to be defined with multiple argument > lists, basically partially applying the function until all of them > are filled. (Sort of like currying, but sort of not.) > > def dec(a, b, foo=bar)(func)(*a, **k): > ...stuff... > return func(*a, **k) > Whhy not create a (meta-)decorator to do this? Something like: def decorator_withargs(decf): def decorator(*args, **kwargs): def decorated(f): return decf(f, *args, **kwargs) return decorated return decorator Then you can write your decorator as: @decorator_withargs def deco(f, a, b, foo='bar'): ...stuff... return f(...) I can't try this now (no python) but it seems to me it should work, even though it makes my head hurt a bit :) OTOH it might be utterly rubbish. -- Arnaud From arno at marooned.org.uk Mon Oct 8 19:42:33 2007 From: arno at marooned.org.uk (Arnaud Delobelle) Date: Mon, 8 Oct 2007 18:42:33 +0100 Subject: [Python-ideas] An easier syntax for writing decorators (& similar things)? In-Reply-To: <35048.194.154.22.39.1191837432.squirrel@www.marooned.org.uk> References: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> <35048.194.154.22.39.1191837432.squirrel@www.marooned.org.uk> Message-ID: On 8 Oct 2007, at 10:57, Arnaud Delobelle wrote: > > On Mon, October 8, 2007 4:33 am, Adam Atlas wrote: >> When writing decorators especially when it's one that needs arguments >> other than the function to be wrapped, it often gets rather ugly... [...] > Whhy not create a (meta-)decorator to do this? Something like: [...] To follow up on my untested suggestion, here's one that is tested: # This metadecorator hasn't changed def decorator_withargs(decf): def decorator(*args, **kwargs): def decorated(f): return decf(f, *args, **kwargs) return decorated return decorator # Here's how to use it to create a decorator @decorator_withargs def mydec(f, before='entering %s', after='%s returns %%s'): before = before % f.__name__ after = after % f.__name__ def decorated(*args, **kwargs): print before result = f(*args, **kwargs) print after % result return result return decorated # Now I can decorate a function with my new decorator @mydec(before='-> %s', after='%s -> %%s') def f(x): print x return x+1 Then >>> f(1) -> f 1 f -> 2 2 -- Arnaud From greg.ewing at canterbury.ac.nz Mon Oct 8 09:45:31 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 08 Oct 2007 20:45:31 +1300 Subject: [Python-ideas] An easier syntax for writing decorators (& similar things)? In-Reply-To: References: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> Message-ID: <4709E01B.3050506@canterbury.ac.nz> Brett Cannon wrote: > On 10/7/07, Adam Atlas wrote: > >>def dec(a, b, foo=bar)(func)(*a, **k): >> ...stuff... >> return func(*a, **k) > > My argument against it is it's ugly and hard to notice. I think it's rather elegant in its own way. There's something exactly analogous in Scheme, where you can write (define (f (g x)) ... ) and it gets macro-expanded into the corresponding nested sequence of lambdas. (The really elegant part about it in Scheme is that it's not a separate feature -- it just automatically falls out of the way define is defined in terms of lambda.) >>I imagine each successive call would >>return an object something like a partial. I would implement it by compiling it exactly as though it were written out as nested defs. -- Greg From adam at atlas.st Wed Oct 10 05:40:04 2007 From: adam at atlas.st (Adam Atlas) Date: Tue, 9 Oct 2007 23:40:04 -0400 Subject: [Python-ideas] An easier syntax for writing decorators (& similar things)? In-Reply-To: References: <554E3DCD-8A1D-4774-98AC-6E1A73AC3B84@atlas.st> Message-ID: On 7 Oct 2007, at 23:49, Brett Cannon wrote: > My argument against it is it's ugly Do you really think so? I think it looks rather elegant, at least in comparison to the indentation-pyramids we have now. > and hard to notice. It is not > obvious unless you really look carefully at the signature to notice > that there are three sets of parentheses there. Spaces would be allowed, of course. Do you think this looks better (less ugly and/or more obvious)? def dec(a, b, foo=bar) (func) (*a, **k): Or maybe they could be separated by commas? I can't decide whether I like that or not... leaning towards not. From jason.orendorff at gmail.com Tue Oct 16 20:56:27 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Tue, 16 Oct 2007 13:56:27 -0500 Subject: [Python-ideas] Concurrent refcounting research Message-ID: GIL-slayers take note. Here are two papers about concurrent reference counting: An On-The-Fly Reference Counting Garbage Collector for Java Levanoni and Petrank, 2001. http://www.cs.technion.ac.il/~erez/Papers/refcount.ps Efficient On-the-Fly Cycle Collection Paz, Bacon, Kolodner, Petrank, and Rajan, 2005. http://www.cs.technion.ac.il/%7Eerez/Papers/CycleCollection.ps "On-the-fly" means the algorithm is neither "fully concurrent" nor "stop the world". Rather, each thread pauses occasionally to do some work. Instead of a GIL, you have a lock that covers this periodic bookkeeping. The details are awfully complex, but there may be insights worth gleaning regardless. Also -- I wrote some stuff at: http://wiki.python.org/moin/GlobalInterpreterLock in the hopes that future "Kill GIL" discussions can start from a better-informed base. -j From rhamph at gmail.com Tue Oct 16 23:26:46 2007 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 16 Oct 2007 15:26:46 -0600 Subject: [Python-ideas] Concurrent refcounting research In-Reply-To: References: Message-ID: On 10/16/07, Jason Orendorff wrote: > GIL-slayers take note. Here are two papers about concurrent reference counting: > > An On-The-Fly Reference Counting Garbage Collector for Java > Levanoni and Petrank, 2001. > http://www.cs.technion.ac.il/~erez/Papers/refcount.ps > > Efficient On-the-Fly Cycle Collection > Paz, Bacon, Kolodner, Petrank, and Rajan, 2005. > http://www.cs.technion.ac.il/%7Eerez/Papers/CycleCollection.ps > > "On-the-fly" means the algorithm is neither "fully concurrent" nor > "stop the world". Rather, each thread pauses occasionally to do some > work. Instead of a GIL, you have a lock that covers this periodic > bookkeeping. > > The details are awfully complex, but there may be insights worth > gleaning regardless. A key assumption of these papers (and presumably most such papers) is that you have a true tracing GC to fall back on, not one bootstrapped from reference counts like CPython has. This means that long-lived or shared objects can disable reference counting entirely, and not face the severe contention it would otherwise create. -- Adam Olsen, aka Rhamphoryncus From christopher.leary at cornell.edu Mon Oct 22 01:29:40 2007 From: christopher.leary at cornell.edu (Christopher D. Leary) Date: Sun, 21 Oct 2007 19:29:40 -0400 Subject: [Python-ideas] Monitor implementation for the stdlib? Message-ID: I was surprised to find that there is no "monitor construct" implementation for the stdlib. I wrote one that I feel is pretty Pythonic, and I'd like it to be torn apart :) It's fairly well documented -- for most standard monitor use cases you can simply inherit the Monitor class and use the Monitor-specific condition variables. Hopefully the accompanying examples will also help to clarify the usage. They're somewhat silly, but get the idea across. As an aside, because this seems to come up in every conversation I've had about monitors in python: if I'm not mistaken, monitors are useful with or without the GIL :) Monitors are nifty tools that make complex synchronization problems somewhat simpler (though more serialized). So far as I understand it, the GIL provides single-bytecode atomicity, and monitor methods are rarely single instructions. Plus, Jython and IronPython don't have a GIL, so I would argue that monitors can still be valuable to "Python the language" even if you won't allow that they can be valuable to "Python the standard implementation". Looking forward to hearing everybody's opinions. Cheers, Chris -------------- next part -------------- A non-text attachment was scrubbed... Name: monitor.py Type: text/x-python Size: 8421 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bridge.py Type: text/x-python Size: 3328 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bar.py Type: text/x-python Size: 4448 bytes Desc: not available URL: From guido at python.org Mon Oct 22 01:57:56 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 21 Oct 2007 16:57:56 -0700 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: Isn't this just syntactic sugar on top of recursive locks and condition variables? What's the big deal apart from conforming to a historically important API? Note that as of python 2.5, the with-statement lets you create critical sections simply by writing with : i.e. the acquire() and release() calls are implicit in the with-statement. --Guido 2007/10/21, Christopher D. Leary : > I was surprised to find that there is no "monitor construct" > implementation for the stdlib. I wrote one that I feel is pretty > Pythonic, and I'd like it to be torn apart :) > > It's fairly well documented -- for most standard monitor use cases you > can simply inherit the Monitor class and use the Monitor-specific > condition variables. Hopefully the accompanying examples will also help > to clarify the usage. They're somewhat silly, but get the idea across. > > As an aside, because this seems to come up in every conversation I've > had about monitors in python: if I'm not mistaken, monitors are useful > with or without the GIL :) > > Monitors are nifty tools that make complex synchronization problems > somewhat simpler (though more serialized). So far as I understand it, > the GIL provides single-bytecode atomicity, and monitor methods are > rarely single instructions. Plus, Jython and IronPython don't have a > GIL, so I would argue that monitors can still be valuable to "Python the > language" even if you won't allow that they can be valuable to "Python > the standard implementation". > > Looking forward to hearing everybody's opinions. > > Cheers, > > Chris > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From christopher.leary at cornell.edu Mon Oct 22 04:02:27 2007 From: christopher.leary at cornell.edu (Christopher D. Leary) Date: Sun, 21 Oct 2007 22:02:27 -0400 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: Though I do think that the with statement is an excellent addition to the language (it avoids much of the ugliness and chance for error in symmetric P and V), I don't believe that it adequately covers the use case that monitors are intended for; namely, encapsulation and abstraction of the mutual exclusion involved in locking around /several data entities/ with /several defined functionalities/. Due to how frequently this pattern is encountered, I think that monitors not only reduce clutter with syntactic sugar (simplicity?), but help encourage a more straightforward way of thinking -- several non re-entrant operations on shared data implies monitor usage, not lock acquisition all over the place. Thoughts? Chris Guido van Rossum wrote: > Isn't this just syntactic sugar on top of recursive locks and > condition variables? What's the big deal apart from conforming to a > historically important API? > > Note that as of python 2.5, the with-statement lets you create > critical sections simply by writing > > with : > > > i.e. the acquire() and release() calls are implicit in the with-statement. > > --Guido > > 2007/10/21, Christopher D. Leary : >> I was surprised to find that there is no "monitor construct" >> implementation for the stdlib. I wrote one that I feel is pretty >> Pythonic, and I'd like it to be torn apart :) >> >> It's fairly well documented -- for most standard monitor use cases you >> can simply inherit the Monitor class and use the Monitor-specific >> condition variables. Hopefully the accompanying examples will also help >> to clarify the usage. They're somewhat silly, but get the idea across. >> >> As an aside, because this seems to come up in every conversation I've >> had about monitors in python: if I'm not mistaken, monitors are useful >> with or without the GIL :) >> >> Monitors are nifty tools that make complex synchronization problems >> somewhat simpler (though more serialized). So far as I understand it, >> the GIL provides single-bytecode atomicity, and monitor methods are >> rarely single instructions. Plus, Jython and IronPython don't have a >> GIL, so I would argue that monitors can still be valuable to "Python the >> language" even if you won't allow that they can be valuable to "Python >> the standard implementation". >> >> Looking forward to hearing everybody's opinions. >> >> Cheers, >> >> Chris >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> >> >> > > From guido at python.org Mon Oct 22 04:17:47 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 21 Oct 2007 19:17:47 -0700 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: 2007/10/21, Christopher D. Leary : > Though I do think that the with statement is an excellent addition to > the language (it avoids much of the ugliness and chance for error in > symmetric P and V), I don't believe that it adequately covers the use > case that monitors are intended for; namely, encapsulation and > abstraction of the mutual exclusion involved in locking around /several > data entities/ with /several defined functionalities/. Locks can do that too of course. It looks like a monitor is similar to 'synchronized' in Java except it appears the default (once inheriting from Monitor) is synchronized, and you need to use a decorator to declare an unsynchronized method. Have I got that right? > Due to how frequently this pattern is encountered, I think that monitors > not only reduce clutter with syntactic sugar (simplicity?), but help > encourage a more straightforward way of thinking -- several non > re-entrant operations on shared data implies monitor usage, not lock > acquisition all over the place. I detect a slight prejudice against lock acquisition in your wording. :-) > Thoughts? It seems to be a matter of granularity. With monitors, one is required to create a new class for each group of variables that require synchronization. With locks, that is not necessary. Which is better depends on the complexity of the group of variables and (especially) the operations. A more "Pythonic" approach would be to implement an @synchronized decorator; Python users are more likely to be familiar with Java terms than with classic Hoare or P and V. (I'm sure someone has already implemented @synchronized, but it hasn't made it into the standard library -- perhaps an indication that its importance is more theoretical than practical, perhaps simply due to a time lag.) --Guido > Chris > > Guido van Rossum wrote: > > Isn't this just syntactic sugar on top of recursive locks and > > condition variables? What's the big deal apart from conforming to a > > historically important API? > > > > Note that as of python 2.5, the with-statement lets you create > > critical sections simply by writing > > > > with : > > > > > > i.e. the acquire() and release() calls are implicit in the with-statement. > > > > --Guido > > > > 2007/10/21, Christopher D. Leary : > >> I was surprised to find that there is no "monitor construct" > >> implementation for the stdlib. I wrote one that I feel is pretty > >> Pythonic, and I'd like it to be torn apart :) > >> > >> It's fairly well documented -- for most standard monitor use cases you > >> can simply inherit the Monitor class and use the Monitor-specific > >> condition variables. Hopefully the accompanying examples will also help > >> to clarify the usage. They're somewhat silly, but get the idea across. > >> > >> As an aside, because this seems to come up in every conversation I've > >> had about monitors in python: if I'm not mistaken, monitors are useful > >> with or without the GIL :) > >> > >> Monitors are nifty tools that make complex synchronization problems > >> somewhat simpler (though more serialized). So far as I understand it, > >> the GIL provides single-bytecode atomicity, and monitor methods are > >> rarely single instructions. Plus, Jython and IronPython don't have a > >> GIL, so I would argue that monitors can still be valuable to "Python the > >> language" even if you won't allow that they can be valuable to "Python > >> the standard implementation". > >> > >> Looking forward to hearing everybody's opinions. > >> > >> Cheers, > >> > >> Chris > >> > >> _______________________________________________ > >> Python-ideas mailing list > >> Python-ideas at python.org > >> http://mail.python.org/mailman/listinfo/python-ideas > >> > >> > >> > > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From santagada at gmail.com Mon Oct 22 05:43:22 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Mon, 22 Oct 2007 00:43:22 -0300 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: Em 21/10/2007, ?s 23:17, Guido van Rossum escreveu: > 2007/10/21, Christopher D. Leary : >> Though I do think that the with statement is an excellent addition to >> the language (it avoids much of the ugliness and chance for error in >> symmetric P and V), I don't believe that it adequately covers the use >> case that monitors are intended for; namely, encapsulation and >> abstraction of the mutual exclusion involved in locking around / >> several >> data entities/ with /several defined functionalities/. > > Locks can do that too of course. Of course, as you know his code is writen using locks > It looks like a monitor is similar to 'synchronized' in Java except it > appears the default (once inheriting from Monitor) is synchronized, > and you need to use a decorator to declare an unsynchronized method. > Have I got that right? Yep, any java class with synchronized is a monitor. > It seems to be a matter of granularity. With monitors, one is required > to create a new class for each group of variables that require > synchronization. With locks, that is not necessary. Which is better > depends on the complexity of the group of variables and (especially) > the operations. > > A more "Pythonic" approach would be to implement an @synchronized > decorator; Python users are more likely to be familiar with Java terms > than with classic Hoare or P and V. > > (I'm sure someone has already implemented @synchronized, but it hasn't > made it into the standard library -- perhaps an indication that its > importance is more theoretical than practical, perhaps simply due to a > time lag.) There is some implementations of @synchronized on the python cookbook at ASPN, but them don't implement monitors, just a way to make a non- reentrant method. I think Christopher implemented monitors that by default have all its methods as if they where java synchronized is because that is most common in python where you don't have the abundance of geters and static methods that java has. I would like to see monitors in the python std library if for no other reason to let concurrency teachers and java fanatics happy. I'm (althought still sitting on my hands about it) wanting to see STM on python. -- Leonardo Santagada From christopher.leary at cornell.edu Mon Oct 22 05:00:27 2007 From: christopher.leary at cornell.edu (Christopher D. Leary) Date: Sun, 21 Oct 2007 23:00:27 -0400 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: <471C124B.9030008@cornell.edu> I didn't mean to imply that locks /couldn't/ model any of the mentioned functionality, just that they weren't the right tool for the job that I was describing (which I believe is frequently occurring in multithreaded applications -- is that in dispute?). I debated putting the unsychronized decorator into the implementation. On one hand, a monitor is a monitor -- it's invariant is mutual exclusion between methods; however, on the other hand you can have related functionality to a "bridge" monitor that doesn't require mutual exclusion, though I figured it should probably be used very sparingly, or the point of a monitor approaches moot -- it just appealed to me more than forcing the equivalent functionality to move to global functions. Perhaps the alternative is better if it makes the monitor concept significantly weaker. That aside, I think the synchronized decorator silly (though I may be wrong, it happens all the time ;). I can envision very few situations where you would only want to make a couple of your methods mutually exclusive and none of the others. In my mind this would imply you're either using poor encapsulation or leaving yourself prone to errors... from a maintainability and understandability perspective I believe that you're losing value by selectively choosing which methods should be synchronized, whereas inheriting Monitor is a clear statement that the instance variables are protected in a thread safe manner. I wholly agree that locks are better suited for some (very complex xor very simple) multithreaded situations. I would think that in the common case where grouping data for thread safety is involved, you'll have very few, disjoint groups, in which case monitors are what you're looking for. I'd be interested in hearing more about your reasoning for synchronized. The monitor "everything is mutually exclusive" seems even /more/ simple to me than "selected functions are mutually exclusive" -- it's certainly less to worry about going forward. Chris Guido van Rossum wrote: > 2007/10/21, Christopher D. Leary : >> Though I do think that the with statement is an excellent addition to >> the language (it avoids much of the ugliness and chance for error in >> symmetric P and V), I don't believe that it adequately covers the use >> case that monitors are intended for; namely, encapsulation and >> abstraction of the mutual exclusion involved in locking around /several >> data entities/ with /several defined functionalities/. > > Locks can do that too of course. > > It looks like a monitor is similar to 'synchronized' in Java except it > appears the default (once inheriting from Monitor) is synchronized, > and you need to use a decorator to declare an unsynchronized method. > Have I got that right? > >> Due to how frequently this pattern is encountered, I think that monitors >> not only reduce clutter with syntactic sugar (simplicity?), but help >> encourage a more straightforward way of thinking -- several non >> re-entrant operations on shared data implies monitor usage, not lock >> acquisition all over the place. > > I detect a slight prejudice against lock acquisition in your wording. :-) > >> Thoughts? > > It seems to be a matter of granularity. With monitors, one is required > to create a new class for each group of variables that require > synchronization. With locks, that is not necessary. Which is better > depends on the complexity of the group of variables and (especially) > the operations. > > A more "Pythonic" approach would be to implement an @synchronized > decorator; Python users are more likely to be familiar with Java terms > than with classic Hoare or P and V. > > (I'm sure someone has already implemented @synchronized, but it hasn't > made it into the standard library -- perhaps an indication that its > importance is more theoretical than practical, perhaps simply due to a > time lag.) > > --Guido > >> Chris >> >> Guido van Rossum wrote: >>> Isn't this just syntactic sugar on top of recursive locks and >>> condition variables? What's the big deal apart from conforming to a >>> historically important API? >>> >>> Note that as of python 2.5, the with-statement lets you create >>> critical sections simply by writing >>> >>> with : >>> >>> >>> i.e. the acquire() and release() calls are implicit in the with-statement. >>> >>> --Guido >>> >>> 2007/10/21, Christopher D. Leary : >>>> I was surprised to find that there is no "monitor construct" >>>> implementation for the stdlib. I wrote one that I feel is pretty >>>> Pythonic, and I'd like it to be torn apart :) >>>> >>>> It's fairly well documented -- for most standard monitor use cases you >>>> can simply inherit the Monitor class and use the Monitor-specific >>>> condition variables. Hopefully the accompanying examples will also help >>>> to clarify the usage. They're somewhat silly, but get the idea across. >>>> >>>> As an aside, because this seems to come up in every conversation I've >>>> had about monitors in python: if I'm not mistaken, monitors are useful >>>> with or without the GIL :) >>>> >>>> Monitors are nifty tools that make complex synchronization problems >>>> somewhat simpler (though more serialized). So far as I understand it, >>>> the GIL provides single-bytecode atomicity, and monitor methods are >>>> rarely single instructions. Plus, Jython and IronPython don't have a >>>> GIL, so I would argue that monitors can still be valuable to "Python the >>>> language" even if you won't allow that they can be valuable to "Python >>>> the standard implementation". >>>> >>>> Looking forward to hearing everybody's opinions. >>>> >>>> Cheers, >>>> >>>> Chris >>>> >>>> _______________________________________________ >>>> Python-ideas mailing list >>>> Python-ideas at python.org >>>> http://mail.python.org/mailman/listinfo/python-ideas >>>> >>>> >>>> >>> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > -- Christopher D. Leary ECE/CS Undergraduate Cornell University C: 914.844.1622 christopher.leary at cornell.edu From guido at python.org Mon Oct 22 05:28:01 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 21 Oct 2007 20:28:01 -0700 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: <471C124B.9030008@cornell.edu> References: <471C124B.9030008@cornell.edu> Message-ID: 2007/10/21, Christopher D. Leary : > I didn't mean to imply that locks /couldn't/ model any of the mentioned > functionality, just that they weren't the right tool for the job that I > was describing (which I believe is frequently occurring in multithreaded > applications -- is that in dispute?). What I'm trying to say (between the lines :-) is that the available tools are IMO good enough and that I don't see the great improvement in convenience you are perceiving with Monitors. > I debated putting the unsychronized decorator into the implementation. > On one hand, a monitor is a monitor -- it's invariant is mutual > exclusion between methods; however, on the other hand you can have > related functionality to a "bridge" monitor that doesn't require mutual > exclusion, though I figured it should probably be used very sparingly, > or the point of a monitor approaches moot -- it just appealed to me more > than forcing the equivalent functionality to move to global functions. > Perhaps the alternative is better if it makes the monitor concept > significantly weaker. Perhaps you should focus on purity, and tell people who want unsynchronized access that they can't use a monitor. :-) > That aside, I think the synchronized decorator silly (though I may be > wrong, it happens all the time ;). Assuming a @synchronized decorator that works like Java's synchronized method, would it still be silly? > I can envision very few situations > where you would only want to make a couple of your methods mutually > exclusive and none of the others. A clear lack of imagination on your part... :-) (Or lack of practical experience?) > In my mind this would imply you're > either using poor encapsulation or leaving yourself prone to errors... > from a maintainability and understandability perspective I believe that > you're losing value by selectively choosing which methods should be > synchronized, whereas inheriting Monitor is a clear statement that the > instance variables are protected in a thread safe manner. I take it you don't like Java much. I wonder though if you realize how easy it is in Python to break through any abstraction? > I wholly agree that locks are better suited for some (very complex xor > very simple) multithreaded situations. I would think that in the common > case where grouping data for thread safety is involved, you'll have very > few, disjoint groups, in which case monitors are what you're looking for. Sounds like a matter of taste to me. > I'd be interested in hearing more about your reasoning for synchronized. > The monitor "everything is mutually exclusive" seems even /more/ simple > to me than "selected functions are mutually exclusive" -- it's certainly > less to worry about going forward. Maybe you should talk to Java's designers. --Guido > Chris > > Guido van Rossum wrote: > > 2007/10/21, Christopher D. Leary : > >> Though I do think that the with statement is an excellent addition to > >> the language (it avoids much of the ugliness and chance for error in > >> symmetric P and V), I don't believe that it adequately covers the use > >> case that monitors are intended for; namely, encapsulation and > >> abstraction of the mutual exclusion involved in locking around /several > >> data entities/ with /several defined functionalities/. > > > > Locks can do that too of course. > > > > It looks like a monitor is similar to 'synchronized' in Java except it > > appears the default (once inheriting from Monitor) is synchronized, > > and you need to use a decorator to declare an unsynchronized method. > > Have I got that right? > > > >> Due to how frequently this pattern is encountered, I think that monitors > >> not only reduce clutter with syntactic sugar (simplicity?), but help > >> encourage a more straightforward way of thinking -- several non > >> re-entrant operations on shared data implies monitor usage, not lock > >> acquisition all over the place. > > > > I detect a slight prejudice against lock acquisition in your wording. :-) > > > >> Thoughts? > > > > It seems to be a matter of granularity. With monitors, one is required > > to create a new class for each group of variables that require > > synchronization. With locks, that is not necessary. Which is better > > depends on the complexity of the group of variables and (especially) > > the operations. > > > > A more "Pythonic" approach would be to implement an @synchronized > > decorator; Python users are more likely to be familiar with Java terms > > than with classic Hoare or P and V. > > > > (I'm sure someone has already implemented @synchronized, but it hasn't > > made it into the standard library -- perhaps an indication that its > > importance is more theoretical than practical, perhaps simply due to a > > time lag.) > > > > --Guido > > > >> Chris > >> > >> Guido van Rossum wrote: > >>> Isn't this just syntactic sugar on top of recursive locks and > >>> condition variables? What's the big deal apart from conforming to a > >>> historically important API? > >>> > >>> Note that as of python 2.5, the with-statement lets you create > >>> critical sections simply by writing > >>> > >>> with : > >>> > >>> > >>> i.e. the acquire() and release() calls are implicit in the with-statement. > >>> > >>> --Guido > >>> > >>> 2007/10/21, Christopher D. Leary : > >>>> I was surprised to find that there is no "monitor construct" > >>>> implementation for the stdlib. I wrote one that I feel is pretty > >>>> Pythonic, and I'd like it to be torn apart :) > >>>> > >>>> It's fairly well documented -- for most standard monitor use cases you > >>>> can simply inherit the Monitor class and use the Monitor-specific > >>>> condition variables. Hopefully the accompanying examples will also help > >>>> to clarify the usage. They're somewhat silly, but get the idea across. > >>>> > >>>> As an aside, because this seems to come up in every conversation I've > >>>> had about monitors in python: if I'm not mistaken, monitors are useful > >>>> with or without the GIL :) > >>>> > >>>> Monitors are nifty tools that make complex synchronization problems > >>>> somewhat simpler (though more serialized). So far as I understand it, > >>>> the GIL provides single-bytecode atomicity, and monitor methods are > >>>> rarely single instructions. Plus, Jython and IronPython don't have a > >>>> GIL, so I would argue that monitors can still be valuable to "Python the > >>>> language" even if you won't allow that they can be valuable to "Python > >>>> the standard implementation". > >>>> > >>>> Looking forward to hearing everybody's opinions. > >>>> > >>>> Cheers, > >>>> > >>>> Chris > >>>> > >>>> _______________________________________________ > >>>> Python-ideas mailing list > >>>> Python-ideas at python.org > >>>> http://mail.python.org/mailman/listinfo/python-ideas > >>>> > >>>> > >>>> > >>> > >> _______________________________________________ > >> Python-ideas mailing list > >> Python-ideas at python.org > >> http://mail.python.org/mailman/listinfo/python-ideas > >> > > > > > > -- > Christopher D. Leary > ECE/CS Undergraduate > Cornell University > C: 914.844.1622 > christopher.leary at cornell.edu > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Mon Oct 22 08:21:56 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 22 Oct 2007 00:21:56 -0600 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: On 10/21/07, Christopher D. Leary wrote: > I was surprised to find that there is no "monitor construct" > implementation for the stdlib. I wrote one that I feel is pretty > Pythonic, and I'd like it to be torn apart :) > > It's fairly well documented -- for most standard monitor use cases you > can simply inherit the Monitor class and use the Monitor-specific > condition variables. Hopefully the accompanying examples will also help > to clarify the usage. They're somewhat silly, but get the idea across. > > As an aside, because this seems to come up in every conversation I've > had about monitors in python: if I'm not mistaken, monitors are useful > with or without the GIL :) > > Monitors are nifty tools that make complex synchronization problems > somewhat simpler (though more serialized). So far as I understand it, > the GIL provides single-bytecode atomicity, and monitor methods are > rarely single instructions. Plus, Jython and IronPython don't have a > GIL, so I would argue that monitors can still be valuable to "Python the > language" even if you won't allow that they can be valuable to "Python > the standard implementation". > > Looking forward to hearing everybody's opinions. My threading extensions to Python (which I still haven't posted) will provide a Monitor class and use it fairly often. There's two major distinctions from what you suggest though: * Deadlocks will be detected and broken automatically * Entering the monitor will be enforced by the language. You'll never need to consult a memory model to determine correctness. This makes it more like Concurrent Pascal's monitors or Erlang's actors than Java's monitors. However, it won't drop a lock on every method automatically. You need a @monitormethod decorator for that. A method is often *NOT* the correct granularity to achieve a thread-safe API! Consider: l = ... some "thread-safe" list ... if l: x = l.pop() There's two method calls, so you cannot guarantee the object hasn't changed between them. I get the impression this is regarded as a major mistake in Java's extensive use of monitors - plastering locks everywhere just isn't good enough. -- Adam Olsen, aka Rhamphoryncus From george.sakkis at gmail.com Mon Oct 22 14:59:46 2007 From: george.sakkis at gmail.com (George Sakkis) Date: Mon, 22 Oct 2007 08:59:46 -0400 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: <91ad5bf80710220559j16275477qae9811315b145177@mail.gmail.com> On 10/22/07, Adam Olsen wrote: > A method is often *NOT* the > correct granularity to achieve a thread-safe API! Consider: > > l = ... some "thread-safe" list ... > if l: > x = l.pop() > > There's two method calls, so you cannot guarantee the object hasn't > changed between them. I get the impression this is regarded as a > major mistake in Java's extensive use of monitors - plastering locks > everywhere just isn't good enough. My Java's kinda rusty but IIRC one can use synchronized blocks on an object to achieve finer granularity. George From rhamph at gmail.com Mon Oct 22 18:22:00 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 22 Oct 2007 10:22:00 -0600 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: <91ad5bf80710220559j16275477qae9811315b145177@mail.gmail.com> References: <91ad5bf80710220559j16275477qae9811315b145177@mail.gmail.com> Message-ID: On 10/22/07, George Sakkis wrote: > On 10/22/07, Adam Olsen wrote: > > > A method is often *NOT* the > > correct granularity to achieve a thread-safe API! Consider: > > > > l = ... some "thread-safe" list ... > > if l: > > x = l.pop() > > > > There's two method calls, so you cannot guarantee the object hasn't > > changed between them. I get the impression this is regarded as a > > major mistake in Java's extensive use of monitors - plastering locks > > everywhere just isn't good enough. > > My Java's kinda rusty but IIRC one can use synchronized blocks on an > object to achieve finer granularity. You need coarser granularity to achieve correctness in this case. -- Adam Olsen, aka Rhamphoryncus From santagada at gmail.com Mon Oct 22 19:56:25 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Mon, 22 Oct 2007 14:56:25 -0300 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: <82FB17DE-CC74-46A2-AF3D-7AB216FF51E6@gmail.com> Em 22/10/2007, ?s 03:21, Adam Olsen escreveu: > My threading extensions to Python (which I still haven't posted) will > provide a Monitor class and use it fairly often. There's two major > distinctions from what you suggest though: When are you planing on doing it? Why not put it on a googlecode svn repository? -- Leonardo Santagada A: Because it breaks the logical sequence of the discussion. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From rhamph at gmail.com Mon Oct 22 19:14:34 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 22 Oct 2007 11:14:34 -0600 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: <82FB17DE-CC74-46A2-AF3D-7AB216FF51E6@gmail.com> References: <82FB17DE-CC74-46A2-AF3D-7AB216FF51E6@gmail.com> Message-ID: On 10/22/07, Leonardo Santagada wrote: > > Em 22/10/2007, ?s 03:21, Adam Olsen escreveu: > > My threading extensions to Python (which I still haven't posted) will > > provide a Monitor class and use it fairly often. There's two major > > distinctions from what you suggest though: > > When are you planing on doing it? I'm working on it now, although I take forever. Major functionality is still missing. > Why not put it on a googlecode svn repository? They're picky about licensing. I don't think I can put a patch to python on there. I will eventually post on python's bug tracker. -- Adam Olsen, aka Rhamphoryncus From george.sakkis at gmail.com Mon Oct 22 20:07:37 2007 From: george.sakkis at gmail.com (George Sakkis) Date: Mon, 22 Oct 2007 14:07:37 -0400 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: <91ad5bf80710220559j16275477qae9811315b145177@mail.gmail.com> Message-ID: <91ad5bf80710221107j59e44300r18c31fb3da72d4ec@mail.gmail.com> On 10/22/07, Adam Olsen wrote: > On 10/22/07, George Sakkis wrote: > > On 10/22/07, Adam Olsen wrote: > > > > > A method is often *NOT* the > > > correct granularity to achieve a thread-safe API! Consider: > > > > > > l = ... some "thread-safe" list ... > > > if l: > > > x = l.pop() > > > > > > There's two method calls, so you cannot guarantee the object hasn't > > > changed between them. I get the impression this is regarded as a > > > major mistake in Java's extensive use of monitors - plastering locks > > > everywhere just isn't good enough. > > > > My Java's kinda rusty but IIRC one can use synchronized blocks on an > > object to achieve finer granularity. > > You need coarser granularity to achieve correctness in this case. Well I was talking from the POV of the enclosing method, i.e. something like: l = ... some "thread-safe" list ... synchronized(I) { if l: x = l.pop() } George From guido at python.org Mon Oct 22 20:27:31 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 22 Oct 2007 11:27:31 -0700 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: <82FB17DE-CC74-46A2-AF3D-7AB216FF51E6@gmail.com> Message-ID: 2007/10/22, Adam Olsen : > On 10/22/07, Leonardo Santagada wrote: > > > > Em 22/10/2007, ?s 03:21, Adam Olsen escreveu: > > > My threading extensions to Python (which I still haven't posted) will > > > provide a Monitor class and use it fairly often. There's two major > > > distinctions from what you suggest though: > > > > When are you planing on doing it? > > I'm working on it now, although I take forever. Major functionality > is still missing. Beware; I think you're close to getting guilty of pre-announcing vaporware. :-) > > Why not put it on a googlecode svn repository? > > They're picky about licensing. I don't think I can put a patch to > python on there. Are you sure? Where do you read this? The way I see it, you can't make something available using the Python license (and for good reason, it's a crappy license even though it's OSI-approved and GPL-compatible), but there's no reason you can't release patches to Python under the Apache 2 license -- after all the PSF suggests that license for all contributions *to* Python as well. > I will eventually post on python's bug tracker. Sounds hardly the place for a variant of the language. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Mon Oct 22 20:39:20 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 22 Oct 2007 12:39:20 -0600 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: <91ad5bf80710221107j59e44300r18c31fb3da72d4ec@mail.gmail.com> References: <91ad5bf80710220559j16275477qae9811315b145177@mail.gmail.com> <91ad5bf80710221107j59e44300r18c31fb3da72d4ec@mail.gmail.com> Message-ID: On 10/22/07, George Sakkis wrote: > On 10/22/07, Adam Olsen wrote: > > On 10/22/07, George Sakkis wrote: > > > On 10/22/07, Adam Olsen wrote: > > > > > > > A method is often *NOT* the > > > > correct granularity to achieve a thread-safe API! Consider: > > > > > > > > l = ... some "thread-safe" list ... > > > > if l: > > > > x = l.pop() > > > > > > > > There's two method calls, so you cannot guarantee the object hasn't > > > > changed between them. I get the impression this is regarded as a > > > > major mistake in Java's extensive use of monitors - plastering locks > > > > everywhere just isn't good enough. > > > > > > My Java's kinda rusty but IIRC one can use synchronized blocks on an > > > object to achieve finer granularity. > > > > You need coarser granularity to achieve correctness in this case. > > Well I was talking from the POV of the enclosing method, i.e. something like: > > l = ... some "thread-safe" list ... > synchronized(I) { > if l: > x = l.pop() > } aka: l = ... some "thread-safe" list ... with l.lock: if l: x = l.pop() But if this "list" doesn't provide a thread-safe API in the first place it has no business providing a lock. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Mon Oct 22 20:45:30 2007 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 22 Oct 2007 12:45:30 -0600 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: <82FB17DE-CC74-46A2-AF3D-7AB216FF51E6@gmail.com> Message-ID: On 10/22/07, Guido van Rossum wrote: > 2007/10/22, Adam Olsen : > > On 10/22/07, Leonardo Santagada wrote: > > > > > > Em 22/10/2007, ?s 03:21, Adam Olsen escreveu: > > > > My threading extensions to Python (which I still haven't posted) will > > > > provide a Monitor class and use it fairly often. There's two major > > > > distinctions from what you suggest though: > > > > > > When are you planing on doing it? > > > > I'm working on it now, although I take forever. Major functionality > > is still missing. > > Beware; I think you're close to getting guilty of pre-announcing vaporware. :-) I've been there so long I have a doormat that reads "welcome vaporware-land". ;) > > > > Why not put it on a googlecode svn repository? > > > > They're picky about licensing. I don't think I can put a patch to > > python on there. > > Are you sure? Where do you read this? The way I see it, you can't make > something available using the Python license (and for good reason, > it's a crappy license even though it's OSI-approved and > GPL-compatible), but there's no reason you can't release patches to > Python under the Apache 2 license -- after all the PSF suggests that > license for all contributions *to* Python as well. Since the patch is based on code using the Python license I'm not sure how the diffs I post will be *completely* under my license. I planned to ask the PSF to help me sort out what *exactly* needed to do wrt licensing before posting my patch. > > I will eventually post on python's bug tracker. > > Sounds hardly the place for a variant of the language. I haven't decided if I should view it as a language variant or as an enhancement. -- Adam Olsen, aka Rhamphoryncus From guido at python.org Mon Oct 22 20:55:28 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 22 Oct 2007 11:55:28 -0700 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: <82FB17DE-CC74-46A2-AF3D-7AB216FF51E6@gmail.com> Message-ID: > > > > Why not put it on a googlecode svn repository? > > > > > > They're picky about licensing. I don't think I can put a patch to > > > python on there. > > > > Are you sure? Where do you read this? The way I see it, you can't make > > something available using the Python license (and for good reason, > > it's a crappy license even though it's OSI-approved and > > GPL-compatible), but there's no reason you can't release patches to > > Python under the Apache 2 license -- after all the PSF suggests that > > license for all contributions *to* Python as well. 2007/10/22, Adam Olsen : > Since the patch is based on code using the Python license I'm not sure > how the diffs I post will be *completely* under my license. Doesn't matter. The PSF license allows you to relicense it. > I planned to ask the PSF to help me sort out what *exactly* needed to > do wrt licensing before posting my patch. Please do; they will be happy to explain this to you. > > > I will eventually post on python's bug tracker. > > > > Sounds hardly the place for a variant of the language. > > I haven't decided if I should view it as a language variant or as an > enhancement. >From the description you mailed me long ago it's clearly a language variant. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Mon Oct 22 21:00:59 2007 From: aahz at pythoncraft.com (Aahz) Date: Mon, 22 Oct 2007 12:00:59 -0700 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: References: Message-ID: <20071022190059.GA14818@panix.com> [BTW, Chris, Guido as BDFL gets a pass on complaints about top-posting, but you're not immune to them. ;-) Please use the standard formatting of mixed quoting as I do below.] On Sun, Oct 21, 2007, Christopher D. Leary wrote: > > Though I do think that the with statement is an excellent addition to > the language (it avoids much of the ugliness and chance for error in > symmetric P and V), I don't believe that it adequately covers the use > case that monitors are intended for; namely, encapsulation and > abstraction of the mutual exclusion involved in locking around /several > data entities/ with /several defined functionalities/. Seems to me that the "standard" Python technique for managing this is message passing to a centralized thread using Queue. What advantages do monitors offer? In any event, you will almost certainly not get monitors into the standard library until they have proven themselves in the wild as a module posted to PyPI. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The best way to get information on Usenet is not to ask a question, but to post the wrong information. From grosser.meister.morti at gmx.net Mon Oct 22 22:48:08 2007 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Mon, 22 Oct 2007 22:48:08 +0200 Subject: [Python-ideas] XMLGenerator: instead of Message-ID: <471D0C88.7050006@gmx.net> Hi. I modified xml.sax.saxutils.XMLGenerator, _xmlplus.sax.saxutils.XMLGenerator, and _xmlplus.sax.saxutils.LexicalXMLGenerator so that they do no longer produce those ugly close tags for empty elements, but use the short version. So instead of you get just . :) I used the version of saxutils.py that is shipped with python 2.5.1. Where do I send such patches to? Download here: http://twoday.tuwien.ac.at/pub/files/python-xml-sax-saxutils (ZIP, 10 KB) -panzi From brett at python.org Mon Oct 22 23:07:45 2007 From: brett at python.org (Brett Cannon) Date: Mon, 22 Oct 2007 14:07:45 -0700 Subject: [Python-ideas] XMLGenerator: instead of In-Reply-To: <471D0C88.7050006@gmx.net> References: <471D0C88.7050006@gmx.net> Message-ID: Create an issue at http://bugs.python.org/ . -Brett On 10/22/07, Mathias Panzenb?ck wrote: > Hi. > > I modified xml.sax.saxutils.XMLGenerator, _xmlplus.sax.saxutils.XMLGenerator, and > _xmlplus.sax.saxutils.LexicalXMLGenerator so that they do no longer produce those > ugly close tags for empty elements, but use the short version. So instead of > you get just . :) > > I used the version of saxutils.py that is shipped with python 2.5.1. > > Where do I send such patches to? > > Download here: > http://twoday.tuwien.ac.at/pub/files/python-xml-sax-saxutils (ZIP, 10 KB) > > -panzi > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From santagada at gmail.com Tue Oct 23 04:46:20 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Mon, 22 Oct 2007 23:46:20 -0300 Subject: [Python-ideas] Monitor implementation for the stdlib? In-Reply-To: <20071022190059.GA14818@panix.com> References: <20071022190059.GA14818@panix.com> Message-ID: <75F826CA-6FFF-4BEA-80D4-8A205FEF4B64@gmail.com> Em 22/10/2007, ?s 16:00, Aahz escreveu: > In any event, you will almost certainly not get monitors into the > standard library until they have proven themselves in the wild as a > module posted to PyPI. This is a great advice for everyone that has code and want it on stdlib, post it somewhere (googlecode or sourceforge comes to mind) package it with distutils (if you make a tarball and an egg everyone is happy), put it on PyPI and I think you can post about it in some key places like python.announce, this thread now (I for one would be willing to test my classes work with it) and some other places like freshmeat or something. -- Leonardo Santagada A: Because it breaks the logical sequence of the discussion. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From lists at cheimes.de Wed Oct 31 11:02:21 2007 From: lists at cheimes.de (Christian Heimes) Date: Wed, 31 Oct 2007 11:02:21 +0100 Subject: [Python-ideas] str(, base=) as complement to int(, base=) Message-ID: Hello! Python's int type has an optional argument base which allows people to specify a base for the conversion of a string to an integer. >>> int('101', 2) 5 >>> int('a', 16) 10 I've sometimes missed a way to reverse the process. How would you like an optional second argument to str() that takes an int from 2 to 36? >>> str(5, 2) '101' >>> str(10, 16) 'a' I know it's not a killer feature but it feels right to have a complement. How do you like the idea? Christian From steven.bethard at gmail.com Wed Oct 31 15:38:41 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 31 Oct 2007 08:38:41 -0600 Subject: [Python-ideas] str(, base=) as complement to int(, base=) In-Reply-To: References: Message-ID: On 10/31/07, Christian Heimes wrote: > Python's int type has an optional argument base which allows people to > specify a base for the conversion of a string to an integer. > > >>> int('101', 2) > 5 > >>> int('a', 16) > 10 > > I've sometimes missed a way to reverse the process. How would you like > an optional second argument to str() that takes an int from 2 to 36? > > >>> str(5, 2) > '101' > >>> str(10, 16) > 'a' > > I know it's not a killer feature but it feels right to have a > complement. How do you like the idea? This was discussed before: http://mail.python.org/pipermail/python-dev/2006-January/059789.html Seemed like people liked the concept, but there was a fair debate about syntax (should it be the str constructor, should it be an int method, etc.) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From rrr at ronadam.com Wed Oct 31 16:04:54 2007 From: rrr at ronadam.com (Ron Adam) Date: Wed, 31 Oct 2007 10:04:54 -0500 Subject: [Python-ideas] str(, base=) as complement to int(, base=) In-Reply-To: References: Message-ID: <47289996.9000304@ronadam.com> Steven Bethard wrote: > On 10/31/07, Christian Heimes wrote: >> Python's int type has an optional argument base which allows people to >> specify a base for the conversion of a string to an integer. >> >>>>> int('101', 2) >> 5 >>>>> int('a', 16) >> 10 >> >> I've sometimes missed a way to reverse the process. How would you like >> an optional second argument to str() that takes an int from 2 to 36? >> >>>>> str(5, 2) >> '101' >>>>> str(10, 16) >> 'a' >> >> I know it's not a killer feature but it feels right to have a >> complement. How do you like the idea? > > This was discussed before: > > http://mail.python.org/pipermail/python-dev/2006-January/059789.html > > Seemed like people liked the concept, but there was a fair debate > about syntax (should it be the str constructor, should it be an int > method, etc.) > > Steve Or should it be a function in the math or string module? Ron From guido at python.org Wed Oct 31 17:49:24 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 31 Oct 2007 09:49:24 -0700 Subject: [Python-ideas] str(, base=) as complement to int(, base=) In-Reply-To: References: Message-ID: It should *definitely* *not* be the str() constructor, which is already overloaded. Remember str() takes arguments of *any* type. Overloading variants on the conversion to string via arguments to str() doesn't scale. On 10/31/07, Steven Bethard wrote: > On 10/31/07, Christian Heimes wrote: > > Python's int type has an optional argument base which allows people to > > specify a base for the conversion of a string to an integer. > > > > >>> int('101', 2) > > 5 > > >>> int('a', 16) > > 10 > > > > I've sometimes missed a way to reverse the process. How would you like > > an optional second argument to str() that takes an int from 2 to 36? > > > > >>> str(5, 2) > > '101' > > >>> str(10, 16) > > 'a' > > > > I know it's not a killer feature but it feels right to have a > > complement. How do you like the idea? > > This was discussed before: > > http://mail.python.org/pipermail/python-dev/2006-January/059789.html > > Seemed like people liked the concept, but there was a fair debate > about syntax (should it be the str constructor, should it be an int > method, etc.) > > Steve > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/)