2.2a2 release next Wednesday

Barry & I are planning to release Python 2.2a2 next Wednesday (August 19), according to the (updated) schedule in PEP 251. Tomorrow (Saturday, August 15) I plan to fork off a short-lived "release branch", from which the release will be done. Nobody but Barry & I should be checking things in on the release branch. We'll selectively merges the trunk into the branch if needed. We'll merge the branch back to the trunk after the release. Anybody who wants something to show up in 2.2a2 would be wise to check it in today or before dawn tomorrow. Please mail me if you need me to wait for something specific. PS Martin: I currently have two failing tests! - test_b1 crashes (in the "float" subtest): Traceback (most recent call last): File "../Lib/test/test_b1.py", line 259, in ? if float(unicode(" \u0663.\u0661\u0664 ")) != 3.14: ValueError: invalid literal for float(): \u0663.\u0661\u0664 - test_format.py fails: u'abc %\\\u3000' % 1 works? ... no Unexpected exceptions.ValueError : "unsupported format character '\\' (0x5c) at index 5" I expect these are casualties of the --disable-unicode checkins. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org>:
OK. Don't wait on this, but I'm going to try to find time to check in my stuff that provides compilerlike framework support for scripts. Code is tested, docs are written. The issue is packaging; I was going to make it a separate ccframe module, but I'm thinking Greg Ewing's suggestion that it should live in fileinput is on balance a good one. But that means I have to merge in the docs. I'll probably get to this tonight. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> To make inexpensive guns impossible to get is to say that you're putting a money test on getting a gun. It's racism in its worst form. -- Roy Innis, president of the Congress of Racial Equality (CORE), 1988

About two weeks ago, Eric Raymond wrote:
Eric didn't merge it with fileinput, but instead checked in a separate module "compilerlike". I have several comments on the code, but my main complaint is about process. This is a random bit of code that got checked in without any kind of discussion about whether it was worth checking into the standard library, and whether this particular implementation was right. There was some discussion afterwards, to which Eric did not respond. Given that Eric apparently doesn't care enough about his code to defend it, I propose to delete it from CVS. I'll do this as part of the 2.2a3 release which (given the encouraging feedback so far) I'll try to do around Sept. 5. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org>:
Now, wait a second! There *was* in fact discussion of this thing beforehand. Several listmembers said they thought it was a good idea. And I have seen *no* discussion *at all* since it was checked in. What is going on here? Is it possible that you are mistaken about the timing of the checkin, and that what you thought was discussion afterwards was discussion before? Or am I somehow missing listmail? As for process issues...I agree that we need better procedures and criteria for what goes into the library. As you know I've made a start on developing same, but my understanding has been that *you* don't think you'll have the bandwidth for it until 2.2 is out. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> As war and government prove, insanity is the most contagious of diseases. -- Edward Abbey

Well, a response at last.
I have now re-read that discussion; it's in the archives starting this message: http://mail.python.org/pipermail/python-dev/2001-August/016629.html There were several suggestions to merge it with fileinput and some suggestions to restructure it. You seem to have ignored these except the criticism on the name "ccframe" (by choosing an even worse name :-).
Your mail was probably broken -- it wouldn't be the first time :-(. There are two posts in the archives that start with a quote from the checkin mail: http://mail.python.org/pipermail/python-dev/2001-August/017131.html http://mail.python.org/pipermail/python-dev/2001-August/017132.html
That's not an excuse for you to check in random bits of code. Some comments on the code: - A framework like this should be structured as a class or set of related classes, not a bunch of functions with function arguments. This would make the documentation easier to read as well; instead of having a bunch of functions you pass in, you customize the framework byu overriding methods. - The name "compilerlike" is a really poor choice (there's nothing compiler-like in the code). - I would like to see failure to open the file handled differently (so the caller can issue decent error message for inaccessible input files without having to catch all IOError exceptions), but again this is a policy issue that should be customizable. - The policy of not writing the output if it's identical to the input should be optional. There are contexts (like when the tool is invoked by a Makefile) where not writing the output could be harmful: if you touch the input without changing it, Make would invoke the tool over and over again because the output doesn't get touched by the tool. Moreover, there seems to be some bugs: if the output is the same as the input, the output file is not written even if a filename transformation was requested (making Make even less happy); when a transformation is specified by a string, an undefined variable 'stem' is used. Hasty work, Eric. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org>:
As have I. All the stuff in this thread was before the checkin; you were in fact mistaken about the timing of most of the discussion.
I did not ignore these suggestions (one that I took was Greg Ward's suggestion that, after all, just throwing an exception was the right thing). And I was in fact planning to merge this thing with fileinput. Then I looked as what would have to be done to the documentation of fileinput -- in fact, I edited together a combined fileinput documentation page. The result was a mess that convinced me that this does indeed need to be a separate module. There wasn't enough coherence between the old fileinput stuff and my entry points to even make the *documentation* look like a logical unit, let alone the code.
In the event, my mail was not broken.
Right...one of which completely misses the point by suggesting that this is a filter framework, and the other one of which is a "me too" basically addressing the naming issue. Guido, you are yourself *notorious* for dismissing naming issues with "that's unimportant" and "we can fix it later". How can you criticize me for doing likewise?
So what, exactly, makes this 'random'? That, Guido, is not a rhetorical question. We don't have any procedures. We don't have any guidelines. We don't have any history of anything but discussing submissions on python-dev before somebody with commit access checks them in. If no -1 votes and the judgment of somebody with commit privileges who has already got a lot of stuff in the library is not sufficient, *what is*? I'm not trying to be difficult here, but this points at a weakness in our way of doing things. I want to play nice, but I can't if I don't know your actual rules. I don't know what *would* have been sufficient if what I did was not. I don't think anyone else does, either.
Some comments on the code:
This is the sort of critique I was looking for two weeks ago, not a bunch of bikeshedding about how the thing should be named.
Yes, I thought of this. There's a reason I didn't do it that way. Method override would work just fine as a way to pass in the filename transformer, but not the data transformer. The problem is this: the driver or "go do it" method of your hypothetical class (the one you'd pass sys.argv[1:]) can't know which overriden method to call in advance, because which one is right would depend on the argument signature of the hook function -- does it take filelike objects, does it take two strings, etc. Actually it's worse than that; two of the cases (the sponge and the line-by-line filtering) aren't even distinguishable by type signature. So, what the driver function could do is step through three method names looking to see which if any is overridden in the user-created subclass. But would that really be a gain in clarity over having three functions in the module? I'm willing to listen if you think the answer is "yes" and want to tell me why, but it didn't seem so to me. There's something else I could have done. I could have required that the hook function use specific unique formal argument names in each of the three cases and then had the driver code use inspect to dispatch among them -- but that seemed even more klugey. Maybe there is a really elegant and low-overhead method of wrapping these functions in a class, and I have just not found it yet. But if so, it is not (as you appear to believe) for lack of looking. If you have an insight that I have missed, I will cheerfully accept instruction on this issue.
- The name "compilerlike" is a really poor choice (there's nothing compiler-like in the code).
No, there isn't. It's called "compilerlike" because it's a framework for making compilerlike interfaces out of functions. But I'm not attached to that name; CompilerFramework or something of that sort would be fine.
Originally the code originally fielded file I/O errors by complaining to stderr and then exiting. At least two respondents argued that it should simply throw an exception and let the caller do policy, and upon reflection I came to agree with this (this is one of those suggestions you thought I was ignoring). I realize it's tempting to try and embed a range of policy options in the module to save time, but unless we can have reasonable confidence that they will cover all important cases I don't judge the complexity overhead to be worth it. Again, I am open to instruction on this.
Interesting point. A better rule, perhaps, would be to suppress writing of output only if both the content *and* the transformed filename are identical -- that would avoid doing a spurious touch on a no-op modification in pace, without confusing Make.
I'll take the hit for this; my test framework should have covered that case and didn't, because I was in a hurry to get in before the freeze. However; I know the other cases work because I'm *using* them. OK, so here's how I see it: 1. I made a minor implementation error with one case; this can be fixed. 2. You were mistaken in believing that (a) there was no discussion or endorsement of the idea before hand, and that (b) I did not defend or justify the design. 3. Some of the respondents simply missed the point; this thing is *not* a framework for creating filters, and shouldn't be named like one or put in the wrong library bin because of it. 4. There is room for technical debate about the interface design, but no choice I'm aware of that is *obviously* better than three functions -- the class-wrapper approach would have unobvious problems doing the hook function dispatch properly. 5. I was trying to do the right thing, but we sorely lack a useful set of norms for what constitutes `good' vs. `bad' librsary checkins. I am actively interested in helping solve problem. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> Non-cooperation with evil is as much a duty as cooperation with good. -- Mohandas Gandhi

[Eric]
Right...one of which completely misses the point by suggesting that this is a filter framework,
True. It's a tad more general than just a filter. How about UnixishTextfileTransformFramework since that captures it's most distinguishing feature (the fact that it follows a popular unix convention). Also hints that it would chew the hell out of a binary file on Windows. [Guido]
[Eric]
Who cares about signatures? Try the attached. - Gordon -------------- Enclosure number 1 ---------------- import sys, os class UnixishFileTransformer: def transformFile(self, infp, outfp): raise NotImplementedError def transformData(self, data): raise NotImplementedError def transformLine(self, line): raise NotImplementedError def transformFilename(self, nm): raise NotImplementedError def run(self, args): if not args: self.fnm = "stdin" self._do_one(sys.stdin, sys.stdout) else: for fnm in args: if fnm == '-': infp = sys.stdin self.fnm = "stdin" else: infp = open(fnm, 'r') self.fnm = fnm tempfile = file + ".~%s-%d~" % (name, os.getpid()) outfp = open(tempfile, "w") try: self._do_one(infp, outfp) except: os.remove(tempfile) raise infp.close() outfp.close() try: newnm = self.transformFilename(self.fnm) except NotImplementedError: if filecmp.cmp(file, tempfile): continue newnm = self.fnm os.remove(self.fnm) os.rename(tempfile, newnm) def _do_one(self, infp, outfp): try: self.transformFile(infp, outfp) except NotImplementedError: try: while 1: line = infp.readline() if not line: break outfp.write(self.transformLine(line)) except NotImplementedError: try: lump = line + infp.read() outfp.write(self.transformData(lump)) except NotImplementedError: raise NotImplementedError, "no implementation of a transform method provided" if __name__ == '__main__': import getopt class T1(UnixishFileTransformer): def transformFilename(self, fnm): return fnm+'.out' def transformFile(self, infp, outfp): while 1: line = infp.readline() if not line: break if line == "\n": outfp.write("------------------------------------------\n") outfp.write(line) class T2(UnixishFileTransformer): def transformFilename(self, fnm): return fnm+'.out2' def transformLine(self, line): return "<" + line[:-1] + ">\n" class T3(UnixishFileTransformer): def transformFilename(self, fnm): return fnm+'.foo' def transformData(self, data): lines = data.split("\n") lines.reverse() return "\n".join(lines) (options, arguments) = getopt.getopt(sys.argv[1:], "fls") for (switch, val) in options: if switch == '-f': t = T1() t.run(arguments) elif switch == '-l': t = T2() t.run(arguments) elif switch == '-s': t = T3() t.run(arguments) else: print "Unknown option."

Gordon McMillan <gmcm@hypernet.com>:
Eh? What's specific to text files in thec design? -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> A wise and frugal government, which shall restrain men from injuring one another, which shall leave them otherwise free to regulate their own pursuits of industry and improvement, and shall not take from the mouth of labor the bread it has earned. This is the sum of good government, and all that is necessary to close the circle of our felicities. -- Thomas Jefferson, in his 1801 inaugural address

No, I was not mistaken about the timing; you must have misunderstood what I said about the timing. When I posted the URL for this thread this afternoon, I knew that it took place before your checkin. I did not see evidence either in the mailing list or in the code that you took any of the advice though.
So, as a matter of process, you should not have checked it in without coming back to the list with your experience.
I am criticizing you for not responding at all to the feedback -- whether it was mistaken or not. That's another violation of process. Why is suggesting it is a filtering framework completely missing the point? If this is not a filter framework, WHAT IS IT?
Absence of -1 votes is not enough. I didn't see any +1 votes -- just suggestions to try a different tack. I happened to be too busy at the time you checked this in to weigh in, but I had a big -1 in my head which I thought was reflected by other comments. Eric, I respect you as a person, but as a Python developer, I don't trust your judgement enough to let you check stuff in without a clear green light from me.
Everybody else who doesn't know the rules for sure starts a discussion, either here or on the patch manager. You are the only one of the 30+ committers who *repeatedly* commits controversial stuff. I'm not saying that the rules are clear enough (they clearly aren't if even you don't get them), but I think there's a better way to get clarity than by acting like a bull in a china cabinet.
I'll respond to this later. First I want you to be clear on the process: commit privileges are not to be used to force an issue. (Admin privileges will be used to force an issue if necessary. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sat, Sep 01, 2001 at 05:52:10PM -0400, Eric S. Raymond wrote:
I've got a couple modules that may or may not be going into Lib. I described the general outline in PEP 268, and will begin developing those modules in nondist/sandbox sometime this week. Despite having commit privs, I'm not about to just toss those modules right into Lib. While there seems to be a very gentle consensus that they be included, they aren't even written. I'm using the PEP to describe the overall design to people so they can provide steering/commentary before coding starts. I'll be using the sandbox to give people a chance to see them as they develop and *before* they go into Lib. Hell... it even gives people a way to *assist*. Once I feel they're "done enough for an alpha release", then I'll post for a final call to move them to Lib. Of course, if we're in the beta time frame by then, then I may have some problems :-) (but they shouldn't go that long) Yes, I could simply write them and check them in. I feel quite comfortable claiming expertise in HTTP-based networking. But an immediate checkin has a very direct perception: "I know what I'm doing and don't need feedback." I've got a lot of respect for the other developers in this forum, and want any feedback they may have. Thus, I'll do what I can to provide that opportunity. [ we're all busy, so I'll get very little, but giving people the *chance* is a good warm&fuzzy and for the hope to get that *one* comment that really slaps me around to realize there is a Better Way ] The point here is: visibility, ability to provide feedback, and a stepwise process for moving modules from inception to Lib integration. It doesn't need to be written. It is simple a social thing, based on respect for your peers. Cheers, -g -- Greg Stein, http://www.lyra.org/

Eric, are you interested in pursuing this discussion further? You stopped replying, but that could be due to the holiday weekend. Anyway, the conclusion seems to be: - You violated the process as commonly understood by checking in a new module without sufficient consensus. - What you checked in is no good; it needs to be redesigned and renamed. - If you absolutely do not want to use the patch manager for this, you can check it in under the nondist/sandbox part of the tree. You can move your code to the sandbox, or I can delete it for you. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I'd add one more thing: there seems to be enough confusion and disagreement that this probably should be a PEP. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine.

Guido van Rossum <guido@python.org>:
Eric, are you interested in pursuing this discussion further? You stopped replying, but that could be due to the holiday weekend.
I've been at Worldcon. Am digging my way out from under now. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> "One of the ordinary modes, by which tyrants accomplish their purposes without resistance, is, by disarming the people, and making it an offense to keep arms." -- Constitutional scholar and Supreme Court Justice Joseph Story, 1840

As for process issues...I agree that we need better procedures and criteria for what goes into the library.
For a start, my current understanding is: - Adding a module to the standard library is a big change. - Modifying a core type or the interpreter loop is a big change. - Breaking compatibility is a big change. - Any big change requires confirmation specifically from Guido. I was pretty surprised to see compilerlike.py go in. Even if i had seen +1s from everyone else on python-dev, i still wouldn't have been comfortable checking it in without hearing from the BDFL. -- ?!ng

"KY" == Ka-Ping Yee <ping@lfw.org> writes:
KY> I was pretty surprised to see compilerlike.py go in. Even if KY> i had seen +1s from everyone else on python-dev, i still KY> wouldn't have been comfortable checking it in without hearing KY> from the BDFL. Yeah, usually the BDFL has to berate you for weeks to get some minor addition, like oh, mimelib, into the standard library <wink>. -Barry (Yes, Guido, I'm working on it! :)

[Barry A. Warsaw]
Yeah, usually the BDFL has to berate you for weeks to get some minor addition, like oh, mimelib, into the standard library <wink>.
Yeah, I usually check something into Tools/scripts/ after 50 people ask for it. Guido never says anything, but I can sense him scowling. Then after a few years he demands to move it into the library. So I do. Then he complains that the exposed API is more suitable for a tool than a library, but that we have more important things to do than twiddle the API. It's a lot of fun to play within the system <wink>! speaking-of-which-i-should-check-combgen.py-into-Tools/scripts-ly y'rs - tim

Guido van Rossum wrote:
+1 Side note to Eric: as a subscriber to python-dev, but not currently a member of the actual development process, I generally don't see checkin messages unless someone else comments on them. I think at a minimum courtesy would suggest that you post an announcement of the checkin when it's feature work that hasn't been PEPed. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine.

I have tried this with 2.2a1: 1. I have not checked the related C code 2. I have not checked the bug report list
Is that meaningful or a problem? I'm trying to understand type/class unification to consider possible jython issues. regards, Samuele Pedroni

This was a 2.2a1 specific problem: in that version, the object class didn't define a setattr dispatch slot, and object.__setattr__ ended up returning a *bound* method of the object *class*. In 2.2a2 this won't be a problem -- the object now defines __setattr__. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org>:
OK. Don't wait on this, but I'm going to try to find time to check in my stuff that provides compilerlike framework support for scripts. Code is tested, docs are written. The issue is packaging; I was going to make it a separate ccframe module, but I'm thinking Greg Ewing's suggestion that it should live in fileinput is on balance a good one. But that means I have to merge in the docs. I'll probably get to this tonight. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> To make inexpensive guns impossible to get is to say that you're putting a money test on getting a gun. It's racism in its worst form. -- Roy Innis, president of the Congress of Racial Equality (CORE), 1988

About two weeks ago, Eric Raymond wrote:
Eric didn't merge it with fileinput, but instead checked in a separate module "compilerlike". I have several comments on the code, but my main complaint is about process. This is a random bit of code that got checked in without any kind of discussion about whether it was worth checking into the standard library, and whether this particular implementation was right. There was some discussion afterwards, to which Eric did not respond. Given that Eric apparently doesn't care enough about his code to defend it, I propose to delete it from CVS. I'll do this as part of the 2.2a3 release which (given the encouraging feedback so far) I'll try to do around Sept. 5. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org>:
Now, wait a second! There *was* in fact discussion of this thing beforehand. Several listmembers said they thought it was a good idea. And I have seen *no* discussion *at all* since it was checked in. What is going on here? Is it possible that you are mistaken about the timing of the checkin, and that what you thought was discussion afterwards was discussion before? Or am I somehow missing listmail? As for process issues...I agree that we need better procedures and criteria for what goes into the library. As you know I've made a start on developing same, but my understanding has been that *you* don't think you'll have the bandwidth for it until 2.2 is out. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> As war and government prove, insanity is the most contagious of diseases. -- Edward Abbey

Well, a response at last.
I have now re-read that discussion; it's in the archives starting this message: http://mail.python.org/pipermail/python-dev/2001-August/016629.html There were several suggestions to merge it with fileinput and some suggestions to restructure it. You seem to have ignored these except the criticism on the name "ccframe" (by choosing an even worse name :-).
Your mail was probably broken -- it wouldn't be the first time :-(. There are two posts in the archives that start with a quote from the checkin mail: http://mail.python.org/pipermail/python-dev/2001-August/017131.html http://mail.python.org/pipermail/python-dev/2001-August/017132.html
That's not an excuse for you to check in random bits of code. Some comments on the code: - A framework like this should be structured as a class or set of related classes, not a bunch of functions with function arguments. This would make the documentation easier to read as well; instead of having a bunch of functions you pass in, you customize the framework byu overriding methods. - The name "compilerlike" is a really poor choice (there's nothing compiler-like in the code). - I would like to see failure to open the file handled differently (so the caller can issue decent error message for inaccessible input files without having to catch all IOError exceptions), but again this is a policy issue that should be customizable. - The policy of not writing the output if it's identical to the input should be optional. There are contexts (like when the tool is invoked by a Makefile) where not writing the output could be harmful: if you touch the input without changing it, Make would invoke the tool over and over again because the output doesn't get touched by the tool. Moreover, there seems to be some bugs: if the output is the same as the input, the output file is not written even if a filename transformation was requested (making Make even less happy); when a transformation is specified by a string, an undefined variable 'stem' is used. Hasty work, Eric. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org>:
As have I. All the stuff in this thread was before the checkin; you were in fact mistaken about the timing of most of the discussion.
I did not ignore these suggestions (one that I took was Greg Ward's suggestion that, after all, just throwing an exception was the right thing). And I was in fact planning to merge this thing with fileinput. Then I looked as what would have to be done to the documentation of fileinput -- in fact, I edited together a combined fileinput documentation page. The result was a mess that convinced me that this does indeed need to be a separate module. There wasn't enough coherence between the old fileinput stuff and my entry points to even make the *documentation* look like a logical unit, let alone the code.
In the event, my mail was not broken.
Right...one of which completely misses the point by suggesting that this is a filter framework, and the other one of which is a "me too" basically addressing the naming issue. Guido, you are yourself *notorious* for dismissing naming issues with "that's unimportant" and "we can fix it later". How can you criticize me for doing likewise?
So what, exactly, makes this 'random'? That, Guido, is not a rhetorical question. We don't have any procedures. We don't have any guidelines. We don't have any history of anything but discussing submissions on python-dev before somebody with commit access checks them in. If no -1 votes and the judgment of somebody with commit privileges who has already got a lot of stuff in the library is not sufficient, *what is*? I'm not trying to be difficult here, but this points at a weakness in our way of doing things. I want to play nice, but I can't if I don't know your actual rules. I don't know what *would* have been sufficient if what I did was not. I don't think anyone else does, either.
Some comments on the code:
This is the sort of critique I was looking for two weeks ago, not a bunch of bikeshedding about how the thing should be named.
Yes, I thought of this. There's a reason I didn't do it that way. Method override would work just fine as a way to pass in the filename transformer, but not the data transformer. The problem is this: the driver or "go do it" method of your hypothetical class (the one you'd pass sys.argv[1:]) can't know which overriden method to call in advance, because which one is right would depend on the argument signature of the hook function -- does it take filelike objects, does it take two strings, etc. Actually it's worse than that; two of the cases (the sponge and the line-by-line filtering) aren't even distinguishable by type signature. So, what the driver function could do is step through three method names looking to see which if any is overridden in the user-created subclass. But would that really be a gain in clarity over having three functions in the module? I'm willing to listen if you think the answer is "yes" and want to tell me why, but it didn't seem so to me. There's something else I could have done. I could have required that the hook function use specific unique formal argument names in each of the three cases and then had the driver code use inspect to dispatch among them -- but that seemed even more klugey. Maybe there is a really elegant and low-overhead method of wrapping these functions in a class, and I have just not found it yet. But if so, it is not (as you appear to believe) for lack of looking. If you have an insight that I have missed, I will cheerfully accept instruction on this issue.
- The name "compilerlike" is a really poor choice (there's nothing compiler-like in the code).
No, there isn't. It's called "compilerlike" because it's a framework for making compilerlike interfaces out of functions. But I'm not attached to that name; CompilerFramework or something of that sort would be fine.
Originally the code originally fielded file I/O errors by complaining to stderr and then exiting. At least two respondents argued that it should simply throw an exception and let the caller do policy, and upon reflection I came to agree with this (this is one of those suggestions you thought I was ignoring). I realize it's tempting to try and embed a range of policy options in the module to save time, but unless we can have reasonable confidence that they will cover all important cases I don't judge the complexity overhead to be worth it. Again, I am open to instruction on this.
Interesting point. A better rule, perhaps, would be to suppress writing of output only if both the content *and* the transformed filename are identical -- that would avoid doing a spurious touch on a no-op modification in pace, without confusing Make.
I'll take the hit for this; my test framework should have covered that case and didn't, because I was in a hurry to get in before the freeze. However; I know the other cases work because I'm *using* them. OK, so here's how I see it: 1. I made a minor implementation error with one case; this can be fixed. 2. You were mistaken in believing that (a) there was no discussion or endorsement of the idea before hand, and that (b) I did not defend or justify the design. 3. Some of the respondents simply missed the point; this thing is *not* a framework for creating filters, and shouldn't be named like one or put in the wrong library bin because of it. 4. There is room for technical debate about the interface design, but no choice I'm aware of that is *obviously* better than three functions -- the class-wrapper approach would have unobvious problems doing the hook function dispatch properly. 5. I was trying to do the right thing, but we sorely lack a useful set of norms for what constitutes `good' vs. `bad' librsary checkins. I am actively interested in helping solve problem. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> Non-cooperation with evil is as much a duty as cooperation with good. -- Mohandas Gandhi

[Eric]
Right...one of which completely misses the point by suggesting that this is a filter framework,
True. It's a tad more general than just a filter. How about UnixishTextfileTransformFramework since that captures it's most distinguishing feature (the fact that it follows a popular unix convention). Also hints that it would chew the hell out of a binary file on Windows. [Guido]
[Eric]
Who cares about signatures? Try the attached. - Gordon -------------- Enclosure number 1 ---------------- import sys, os class UnixishFileTransformer: def transformFile(self, infp, outfp): raise NotImplementedError def transformData(self, data): raise NotImplementedError def transformLine(self, line): raise NotImplementedError def transformFilename(self, nm): raise NotImplementedError def run(self, args): if not args: self.fnm = "stdin" self._do_one(sys.stdin, sys.stdout) else: for fnm in args: if fnm == '-': infp = sys.stdin self.fnm = "stdin" else: infp = open(fnm, 'r') self.fnm = fnm tempfile = file + ".~%s-%d~" % (name, os.getpid()) outfp = open(tempfile, "w") try: self._do_one(infp, outfp) except: os.remove(tempfile) raise infp.close() outfp.close() try: newnm = self.transformFilename(self.fnm) except NotImplementedError: if filecmp.cmp(file, tempfile): continue newnm = self.fnm os.remove(self.fnm) os.rename(tempfile, newnm) def _do_one(self, infp, outfp): try: self.transformFile(infp, outfp) except NotImplementedError: try: while 1: line = infp.readline() if not line: break outfp.write(self.transformLine(line)) except NotImplementedError: try: lump = line + infp.read() outfp.write(self.transformData(lump)) except NotImplementedError: raise NotImplementedError, "no implementation of a transform method provided" if __name__ == '__main__': import getopt class T1(UnixishFileTransformer): def transformFilename(self, fnm): return fnm+'.out' def transformFile(self, infp, outfp): while 1: line = infp.readline() if not line: break if line == "\n": outfp.write("------------------------------------------\n") outfp.write(line) class T2(UnixishFileTransformer): def transformFilename(self, fnm): return fnm+'.out2' def transformLine(self, line): return "<" + line[:-1] + ">\n" class T3(UnixishFileTransformer): def transformFilename(self, fnm): return fnm+'.foo' def transformData(self, data): lines = data.split("\n") lines.reverse() return "\n".join(lines) (options, arguments) = getopt.getopt(sys.argv[1:], "fls") for (switch, val) in options: if switch == '-f': t = T1() t.run(arguments) elif switch == '-l': t = T2() t.run(arguments) elif switch == '-s': t = T3() t.run(arguments) else: print "Unknown option."

Gordon McMillan <gmcm@hypernet.com>:
Eh? What's specific to text files in thec design? -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> A wise and frugal government, which shall restrain men from injuring one another, which shall leave them otherwise free to regulate their own pursuits of industry and improvement, and shall not take from the mouth of labor the bread it has earned. This is the sum of good government, and all that is necessary to close the circle of our felicities. -- Thomas Jefferson, in his 1801 inaugural address

No, I was not mistaken about the timing; you must have misunderstood what I said about the timing. When I posted the URL for this thread this afternoon, I knew that it took place before your checkin. I did not see evidence either in the mailing list or in the code that you took any of the advice though.
So, as a matter of process, you should not have checked it in without coming back to the list with your experience.
I am criticizing you for not responding at all to the feedback -- whether it was mistaken or not. That's another violation of process. Why is suggesting it is a filtering framework completely missing the point? If this is not a filter framework, WHAT IS IT?
Absence of -1 votes is not enough. I didn't see any +1 votes -- just suggestions to try a different tack. I happened to be too busy at the time you checked this in to weigh in, but I had a big -1 in my head which I thought was reflected by other comments. Eric, I respect you as a person, but as a Python developer, I don't trust your judgement enough to let you check stuff in without a clear green light from me.
Everybody else who doesn't know the rules for sure starts a discussion, either here or on the patch manager. You are the only one of the 30+ committers who *repeatedly* commits controversial stuff. I'm not saying that the rules are clear enough (they clearly aren't if even you don't get them), but I think there's a better way to get clarity than by acting like a bull in a china cabinet.
I'll respond to this later. First I want you to be clear on the process: commit privileges are not to be used to force an issue. (Admin privileges will be used to force an issue if necessary. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sat, Sep 01, 2001 at 05:52:10PM -0400, Eric S. Raymond wrote:
I've got a couple modules that may or may not be going into Lib. I described the general outline in PEP 268, and will begin developing those modules in nondist/sandbox sometime this week. Despite having commit privs, I'm not about to just toss those modules right into Lib. While there seems to be a very gentle consensus that they be included, they aren't even written. I'm using the PEP to describe the overall design to people so they can provide steering/commentary before coding starts. I'll be using the sandbox to give people a chance to see them as they develop and *before* they go into Lib. Hell... it even gives people a way to *assist*. Once I feel they're "done enough for an alpha release", then I'll post for a final call to move them to Lib. Of course, if we're in the beta time frame by then, then I may have some problems :-) (but they shouldn't go that long) Yes, I could simply write them and check them in. I feel quite comfortable claiming expertise in HTTP-based networking. But an immediate checkin has a very direct perception: "I know what I'm doing and don't need feedback." I've got a lot of respect for the other developers in this forum, and want any feedback they may have. Thus, I'll do what I can to provide that opportunity. [ we're all busy, so I'll get very little, but giving people the *chance* is a good warm&fuzzy and for the hope to get that *one* comment that really slaps me around to realize there is a Better Way ] The point here is: visibility, ability to provide feedback, and a stepwise process for moving modules from inception to Lib integration. It doesn't need to be written. It is simple a social thing, based on respect for your peers. Cheers, -g -- Greg Stein, http://www.lyra.org/

Eric, are you interested in pursuing this discussion further? You stopped replying, but that could be due to the holiday weekend. Anyway, the conclusion seems to be: - You violated the process as commonly understood by checking in a new module without sufficient consensus. - What you checked in is no good; it needs to be redesigned and renamed. - If you absolutely do not want to use the patch manager for this, you can check it in under the nondist/sandbox part of the tree. You can move your code to the sandbox, or I can delete it for you. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
I'd add one more thing: there seems to be enough confusion and disagreement that this probably should be a PEP. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine.

Guido van Rossum <guido@python.org>:
Eric, are you interested in pursuing this discussion further? You stopped replying, but that could be due to the holiday weekend.
I've been at Worldcon. Am digging my way out from under now. -- <a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a> "One of the ordinary modes, by which tyrants accomplish their purposes without resistance, is, by disarming the people, and making it an offense to keep arms." -- Constitutional scholar and Supreme Court Justice Joseph Story, 1840

As for process issues...I agree that we need better procedures and criteria for what goes into the library.
For a start, my current understanding is: - Adding a module to the standard library is a big change. - Modifying a core type or the interpreter loop is a big change. - Breaking compatibility is a big change. - Any big change requires confirmation specifically from Guido. I was pretty surprised to see compilerlike.py go in. Even if i had seen +1s from everyone else on python-dev, i still wouldn't have been comfortable checking it in without hearing from the BDFL. -- ?!ng

"KY" == Ka-Ping Yee <ping@lfw.org> writes:
KY> I was pretty surprised to see compilerlike.py go in. Even if KY> i had seen +1s from everyone else on python-dev, i still KY> wouldn't have been comfortable checking it in without hearing KY> from the BDFL. Yeah, usually the BDFL has to berate you for weeks to get some minor addition, like oh, mimelib, into the standard library <wink>. -Barry (Yes, Guido, I'm working on it! :)

[Barry A. Warsaw]
Yeah, usually the BDFL has to berate you for weeks to get some minor addition, like oh, mimelib, into the standard library <wink>.
Yeah, I usually check something into Tools/scripts/ after 50 people ask for it. Guido never says anything, but I can sense him scowling. Then after a few years he demands to move it into the library. So I do. Then he complains that the exposed API is more suitable for a tool than a library, but that we have more important things to do than twiddle the API. It's a lot of fun to play within the system <wink>! speaking-of-which-i-should-check-combgen.py-into-Tools/scripts-ly y'rs - tim

Guido van Rossum wrote:
+1 Side note to Eric: as a subscriber to python-dev, but not currently a member of the actual development process, I generally don't see checkin messages unless someone else comments on them. I think at a minimum courtesy would suggest that you post an announcement of the checkin when it's feature work that hasn't been PEPed. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista I don't really mind a person having the last whine, but I do mind someone else having the last self-righteous whine.

Guido wrote:
I propose to delete it from CVS.
+1 (delete or move to the non-dist sandbox)
I'll do this as part of the 2.2a3 release which (given the encouraging feedback so far) I'll try to do around Sept. 5.
</F>

I have tried this with 2.2a1: 1. I have not checked the related C code 2. I have not checked the bug report list
Is that meaningful or a problem? I'm trying to understand type/class unification to consider possible jython issues. regards, Samuele Pedroni

This was a 2.2a1 specific problem: in that version, the object class didn't define a setattr dispatch slot, and object.__setattr__ ended up returning a *bound* method of the object *class*. In 2.2a2 this won't be a problem -- the object now defines __setattr__. --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (10)
-
aahz@rahul.net
-
barry@zope.com
-
Eric S. Raymond
-
Fredrik Lundh
-
Gordon McMillan
-
Greg Stein
-
Guido van Rossum
-
Ka-Ping Yee
-
Samuele Pedroni
-
Tim Peters