This fixes a typo in CVS and adds a common handler for file logging: Index: Twisted/twisted/web2/log.py =================================================================== --- Twisted/twisted/web2/log.py (revision 15644) +++ Twisted/twisted/web2/log.py (working copy) @@ -137,7 +137,7 @@ request = eventDict['request'] response = eventDict['response'] loginfo = eventDict['loginfo'] - firstLine = '%s %s HTTP/%' %( + firstLine = '%s %s HTTP/%s' %( request.method, request.uri, '.'.join([str(x) for x in request.clientproto])) @@ -166,6 +166,13 @@ tlog.removeObserver(self.emit) +class FileAccessLoggingObserver(BaseCommonAccessLoggingObserver): + def __init__(self, logpath): + self.f = file(logpath, 'a', 1) + + def logMessage(self, message): + self.f.write(message + '\n') + class DefaultCommonAccessLoggingObserver(BaseCommonAccessLoggingObserver): """Log requests to default twisted logfile.""" def logMessage(self, message):
On Fri, 20 Jan 2006 03:35:38 +0100, Andrea Arcangeli
This fixes a typo in CVS and adds a common handler for file logging:
Andrea, Thanks for the patch. Could you attach it to an issue at http://twistedmatrix.com/bugs/? Also, patches with unit tests are likely to be applied more quickly than those without. Jean-Paul
On Thu, Jan 19, 2006 at 10:31:20PM -0500, Jean-Paul Calderone wrote:
On Fri, 20 Jan 2006 03:35:38 +0100, Andrea Arcangeli
wrote: This fixes a typo in CVS and adds a common handler for file logging:
Andrea,
Thanks for the patch. Could you attach it to an issue at http://twistedmatrix.com/bugs/? Also, patches with unit tests are likely to be applied more quickly than those without.
My test suite is the CPUShare server code, I tested it. This is a trivial patch and I hope I can be applied right away without special requirements (a bug was checked into trunk and this fixes it, plus it adds one more class that nobody can be using yet, so it can't make things worse). Or at least I've not nearly the time to write testsuites for such trivial fixes, nor to attach them somewhere with a web browser, sorry. I hoped I could contribute despite significant constraints, I'm sorry if that's not the case. I think development of twisted is too slow if it requires these formalities (especially given this is a web2 _unstable_ branch, I didn't touch anything else in my patch). So I'm going to fork twisted into a private twisted-CPUShare branch for my own server use where I won't have to waste time to fix bugs and, I'll keep merging stuff from trunk as long as it makes sense. Ideally my branch will be the same as trunk but I don't care anymore if that's not the case. I'm going to write right now a prorprietary CPUShare guard with caching of the cookies in ram, and real storage of the cookies in sql. This way restarting the server won't logout users like it happens currently with nevow, which is unacceptable for the long term. The login mechanism of twisted is overkill complicated and it provides zero benefits, so I'll simplify it with the library-model as usual, while I solve the harddisk persistence at the same time (I'll stop using all those portal/realm nosense, where I even had to hack the logout with a local function because it was missing parameters to the callback). From twisted.web2 I only need the functionality to read cookies which seems doable with the http_header response, I don't want to depend on anything more complicated then that. I can publish that guard code once it works if there's interest, but only by email, I've no time for anything else sorry. I hope to have the basic login working before the end of the weekend (then next weekend I'll rewrite all the forms and rendering with Cheetah and FormKit). If something goes wrong in this process I'll drop twisted entirely from the server side since I've wasted enough time already (I can still use it on the client side since it worked well there, there's no http involved in the client side after all and furthermore there are no smp scalability concerns on the client side).
On Jan 20, 2006, at 1:36 PM, Andrea Arcangeli wrote:
My test suite is the CPUShare server code, I tested it.
That is not a unit test. And if you don't care if things keep working, that's okay. However, people generally _do_ like things to continue working. That's what unit tests are for. The logging code has worked in the past, but, as seen by the brokenness currently existing in svn, not having a test means someone may break it and nobody may notice. That's bad. Wouldn't you like to fix that?
This is a trivial patch and I hope I can be applied right away without special requirements (a bug was checked into trunk and this fixes it, plus it adds one more class that nobody can be using yet, so it can't make things worse)
Fixing a bug without adding a test makes things slightly better, but fixing a bug *and* adding a test makes things much better. It'd be nice if you could help out by writing a test, even though it doesn't currently exist. However, even without that, yes, the bugfix will be made. Thanks for reporting it. However, committing a new untested API to trunk makes things overall potentially worse not better, as it increases the likelihood of bugs being added unnoticed. Yes, I've been guilty of doing this, but it's not a _good_ thing. James
On Fri, 20 Jan 2006 19:36:21 +0100, Andrea Arcangeli
On Thu, Jan 19, 2006 at 10:31:20PM -0500, Jean-Paul Calderone wrote:
Andrea,
Thanks for the patch. Could you attach it to an issue at http://twistedmatrix.com/bugs/? Also, patches with unit tests are likely to be applied more quickly than those without.
My test suite is the CPUShare server code, I tested it.
James put this pretty well: if you fixed it this one time, how do we know that this won't just be reverted and broken again by some other patch? We have tests for a reason. We're not going to run CPUShare every time to make sure that it didn't break again.
I hope I can be applied right away without special requirements
This is not a special requirement. All patches are required to come with unit tests - JP's response is repeated so often, it was probably just some text he keeps in his clipboard whenever he is reading email. You are asking for special consideration which, I might add, is more likely to be afforded to those who have a good track-record of writing tests and filing bugs. You've contributed patches, but always free-form on the mailing list with little explanation and no tests - in some cases, that's more work to figure out than fixing the bug myself.
Or at least I've not nearly the time to write testsuites for such trivial fixes,
Maybe if you did, my kernel wouldn't crash once a week... :)
nor to attach them somewhere with a web browser, sorry.
Yet, you spent *all this time* in your email client. How is that easier? This message contained 5x as much text as would have been required in the bug report, probably more than would have been in the test and the report combined.
I hoped I could contribute despite significant constraints, I'm
You are not special. We are *all* operating under significant time constraints for working on Twisted.
sorry if that's not the case. I think development of twisted is too slow if it requires these formalities (especially given this is a web2 _unstable_ branch, I didn't touch anything else in my patch).
At this point, and I realize this may not be intentional, you are actually contributing negatively. The "formalities" are in place to speed up the development process. Without tests, we'd be spending twice or three times as much time reviewing every patch, and preparing for a release would require acceptance testing from dozens or hundreds of people.
So I'm going to fork twisted into a private twisted-CPUShare branch for my own server use where I won't have to waste time to fix bugs and, I'll keep merging stuff from trunk as long as it makes sense.
You remember that 'epsilon' thing you objected to? That is effectively Divmod doing exactly this. It's a great idea. Please do it!
This way restarting the server won't logout users like it happens currently with nevow, which is unacceptable for the long term.
Libraries provide mechanism, user code provides policy. Nevow did not want to specify a required persistence mechanism for sessions, so it is up to the user to implement that - but it has specific hooks (*documented*, *tested* specific hooks, even) which support this use case. There are at least two examples of this: the old Quotient repository, at http://divmod.org/svn/Quotient/trunk/quotient/web/sessionpersist.py and one in the new Mantissa, at http://divmod.org/svn/Divmod/trunk/Mantissa/xmantissa/websession.py. Both of these use the guard provided by nevow.
I'll simplify it with the library-model as usual
I already asked you to stop trolling.
(I'll stop using all those portal/realm nosense,
I'm sorry you don't understand cred, but that doesn't make it "nonsense". Care to provide a specific criticism?
where I even had to hack the logout with a local function because it was missing parameters to the callback).
(By the way, this doesn't count as a "specific criticism" - the convention of using closures in Lisp to accomplish this probably predates the programming career of anyone who posts to this list, and while it wasn't available in early releases of Python, bound methods were used to accomplish the same effect.)
I've no time for anything else sorry.
Why do you assume that everyone else has so much time, to write tests for your code, to file your bugs for you? For that matter, where does all this time for posting to mailing lists come from?
I'll drop twisted entirely from the server side since I've wasted enough time already
You've certainly wasted enough of everyone's time at this point. If you are going to keep using Twisted, could you direct your energies towards more positive contributions? Your bugfixes are welcome enough, but >1 post per day about the "library model" isn't helping anybody, least of all you.
glyph@divmod.com schrieb:
On Fri, 20 Jan 2006 19:36:21 +0100, Andrea Arcangeli
wrote: On Thu, Jan 19, 2006 at 10:31:20PM -0500, Jean-Paul Calderone wrote:
Andrea,
Thanks for the patch. Could you attach it to an issue at http://twistedmatrix.com/bugs/? Also, patches with unit tests are likely to be applied more quickly than those without.
My test suite is the CPUShare server code, I tested it.
James put this pretty well: if you fixed it this one time, how do we know that this won't just be reverted and broken again by some other patch? We have tests for a reason. We're not going to run CPUShare every time to make sure that it didn't break again.
I hope I can be applied right away without special requirements
This is not a special requirement. All patches are required to come with unit tests - JP's response is repeated so often, it was probably just some text he keeps in his clipboard whenever he is reading email. You are asking for special consideration which, I might add, is more likely to be afforded to those who have a good track-record of writing tests and filing bugs. You've contributed patches, but always free-form on the mailing list with little explanation and no tests - in some cases, that's more work to figure out than fixing the bug myself.
Or at least I've not nearly the time to write testsuites for such trivial fixes,
Maybe if you did, my kernel wouldn't crash once a week... :)
nor to attach them somewhere with a web browser, sorry.
Yet, you spent *all this time* in your email client. How is that easier? This message contained 5x as much text as would have been required in the bug report, probably more than would have been in the test and the report combined.
I hoped I could contribute despite significant constraints, I'm
You are not special. We are *all* operating under significant time constraints for working on Twisted.
sorry if that's not the case. I think development of twisted is too slow if it requires these formalities (especially given this is a web2 _unstable_ branch, I didn't touch anything else in my patch).
At this point, and I realize this may not be intentional, you are actually contributing negatively. The "formalities" are in place to speed up the development process. Without tests, we'd be spending twice or three times as much time reviewing every patch, and preparing for a release would require acceptance testing from dozens or hundreds of people.
So I'm going to fork twisted into a private twisted-CPUShare branch for my own server use where I won't have to waste time to fix bugs and, I'll keep merging stuff from trunk as long as it makes sense.
You remember that 'epsilon' thing you objected to? That is effectively Divmod doing exactly this. It's a great idea. Please do it!
This is only a great idea since patches tend to rot in the bugtracker. I've added a patch for sending new style classes via pb at the end of 2003. 10 months later this has been assigned, in march 2005 db3l asks why this patch hasn't been applied (he's using it successfully). In April things start to fly: tests are being written, code is committed (not my simple 3 line diff, something else). However, my test program still doesn't work. In the end it's fixed [http://twistedmatrix.com/bugs/issue426] So, maybe you should check your "formalities" that speed up the development process. I guess they aren't working that good. It doesn't make sense for everyone to manage their own patched twisted, fixing the same bugs... - Ralf
On Fri, Jan 20, 2006 at 10:42:44PM +0100, Ralf Schmitt wrote:
This is only a great idea since patches tend to rot in the bugtracker. I've added a patch for sending new style classes via pb at the end of 2003. 10 months later this has been assigned, in march 2005 db3l asks why this patch hasn't been applied (he's using it successfully).
The same happened for the ssl exceptions.
So, maybe you should check your "formalities" that speed up the development process. I guess they aren't working that good.
Agreed.
It doesn't make sense for everyone to manage their own patched twisted, fixing the same bugs...
I can consider exporting my branch instead of keeping it private. I use mercurial, so development wouldn't be centralized. I could pull from anybody else branch and anybody could pull from mine. I now understand Rogers's choice, but I think a stable branch without formalities could give more confidence (it does for me, and if that fails too, I'll take rogers path). http://twistedmatrix.com/pipermail/twisted-python/2005-May/010300.html
On Fri, 20 Jan 2006 22:42:44 +0100, Ralf Schmitt
glyph@divmod.com schrieb:
You remember that 'epsilon' thing you objected to? That is effectively Divmod doing exactly this. It's a great idea. Please do it!
This is only a great idea since patches tend to rot in the bugtracker.
Most Divmod team members are on the core development team for Twisted, and we *only* put patches in Epsilon which have *already* been applied to Twisted. I've said this at least twice in this thread already. It's a good idea because there is *work* associated with releasing Twisted, work which neither you nor certain other posters seem willing to do.
I've added a patch for sending new style classes via pb at the end of 2003.
... with no unit tests
10 months later this has been assigned, in march 2005 db3l asks why this patch hasn't been applied (he's using it successfully).
... with no unit tests
In April things start to fly: tests are being written,
Here's the causality: "In April, unit tests are written, and things start to fly." I won't promise that all patches in the tracker will be applied immediately, even with appropriate tests - after all, review takes some work too, and as I keep repeating, we are short on manpower - but they certainly rot less in the tracker than they would if they were just some random mailing list post. That bug report provided a hub for quite a bit of discussion, patches, etc, whereas the same thing as a mailing list thread might have been resurrected 5 or 6 times with no actual work being done. More importantly, if the peanut gallery ever decides to start going through the tracker and fixing things rather than trolling the various lists, the presence of your bug report there will give them some meat to work on. The last thing a contributor who is strapped for time wants to do is come to a project and read every message in the last 6 years of archives in reverse-date order, looking for something to work on.
However, my test program still doesn't work.
You almost certainly would have had a better experience if you had submitted it in the form of a unit test. Trial currently has a feature which allows you to return a Deferred from a unit test. The code you submitted could have been several lines *shorter* and been in the correct format; the deferred created by callRemote effectively terminates the test. Even if this code was submitted before Trial added that feature, there are tests for PB which hit the network that you could have cribbed from.
So, maybe you should check your "formalities" that speed up the development process. I guess they aren't working that good.
http://www.gamesfromwithin.com/articles/0503/000078.html """ Here's the interesting part, though: I claim that once you're comfortable with TDD and you've applied it to a large amount of code, your development speed will actually be faster than it would be without TDD. So not only you get all the other benefits, but things will go faster and more smoothly. How's that possible? """ All you are looking at is the speed to apply your one patch - Twisted's overall development speed includes the time taken to fix the bugs introduced by such patches. It may not contain any - but, without tests, how do we know?
It doesn't make sense for everyone to manage their own patched twisted, fixing the same bugs...
You're right about that. Like Divmod, anyone maintaining a local fork because they depend on bugfixes should submit those bugfixes, with tests, to the Twisted project **first**. Finally, this policy isn't going to change. For every minor inconvenience it has caused you, it has saved Twisted contributors days or weeks of work. Whining about it just convinces me that people who don't write tests are whiners who produce buggy code, and who should therefore be ignored. Is this the image you are trying to project?
glyph@divmod.com schrieb:
I've added a patch for sending new style classes via pb at the end of 2003.
... with no unit tests
Come on. I added a simple test program. If someone told me to write unit tests for it, I would have done it. But there has just been no reaction for 10 months. Do the twisted developers only start fixing bugs if the bug reporter writes a unit test revealing that bug?
10 months later this has been assigned, in march 2005 db3l asks why this patch hasn't been applied (he's using it successfully).
... with no unit tests
In April things start to fly: tests are being written,
Here's the causality:
"In April, unit tests are written, and things start to fly."
I won't promise that all patches in the tracker will be applied immediately, even with appropriate tests - after all, review takes some work too, and as I keep repeating, we are short on manpower - but they certainly rot less in the tracker than they would if they were just some random mailing list post. That bug report provided a hub for quite a bit of discussion, patches, etc, whereas the same thing as a mailing list thread might have been resurrected 5 or 6 times with no actual work being done.
ACK
More importantly, if the peanut gallery ever decides to start going through the tracker and fixing things rather than trolling the various lists, the presence of your bug report there will give them some meat to work on. The last thing a contributor who is strapped for time wants to do is come to a project and read every message in the last 6 years of archives in reverse-date order, looking for something to work on.
I'm not trolling. Reporting bugs is also a form of contribution, people spent time on doing it, even sent a patch, and get ignored. [http://twistedmatrix.com/bugs/issue392]. And you start insulting them (btw. what is peanut gallery?), instead of being glad that they give back their feedback?
However, my test program still doesn't work.
You almost certainly would have had a better experience if you had submitted it in the form of a unit test.
My experience was fine. We were already using the patched version. But, to repeat myself: I would have written unit tests if someone told me to do it.
So, maybe you should check your "formalities" that speed up the development process. I guess they aren't working that good.
http://www.gamesfromwithin.com/articles/0503/000078.html
""" Here's the interesting part, though: I claim that once you're comfortable with TDD and you've applied it to a large amount of code, your development speed will actually be faster than it would be without TDD. So not only you get all the other benefits, but things will go faster and more smoothly. How's that possible? """
All you are looking at is the speed to apply your one patch - Twisted's overall development speed includes the time taken to fix the bugs introduced by such patches. It may not contain any - but, without tests, how do we know?
I look at the speed of fixing bugs. And in this particular case it took 18 months or so to fix it. I have added other bug reports, without getting any reaction. IMHO, having working code without tests is better than having buggy code without tests (well, one might argue that it's the same...).
It doesn't make sense for everyone to manage their own patched twisted, fixing the same bugs...
You're right about that. Like Divmod, anyone maintaining a local fork because they depend on bugfixes should submit those bugfixes, with tests, to the Twisted project **first**.
I admit that it's nice for the twisted developers to have bug reports in the form of failing unit tests. But you can't expect everyone to write them.
Finally, this policy isn't going to change. For every minor inconvenience it has caused you, it has saved Twisted contributors days or weeks of work. Whining about it just convinces me that people who don't write tests are whiners who produce buggy code, and who should therefore be ignored. Is this the image you are trying to project?
I'm not whining (actually I write unit tests at work). We use twisted at our company and appreciate the work that has gone into it. But one thinks about saving that work of reporting bugs, when they just aren't dealt with. - Ralf
On Sat, 21 Jan 2006 01:48:33 +0100, Ralf Schmitt
glyph@divmod.com schrieb:
I've added a patch for sending new style classes via pb at the end of 2003.
... with no unit tests Come on. I added a simple test program. If someone told me to write unit tests for it, I would have done it. But there has just been no reaction for 10 months.
Yes. This is a serious problem. I just wrote another post which mentions it. I hope that it does not happen in the future. I misunderstood your first post and thought that you were aware of the requirement and chose to disregard it - I'm sorry.
Do the twisted developers only start fixing bugs if the bug reporter writes a unit test revealing that bug?
No. I could probably argue for a while over whether the bug you submitted was really high-priority; subtle issues opened by the patch; some still-unresolved problems with PB and new-style classes. It's all pretty much moot, though, and offtopic for this list. I remember having several in-person exchanges about that _particular_ patch, none of which were recorded in the tracker. That, also, is a problem, but we have so few in-person exchanges it's hardly worth worrying about. The context of the unit-tests discussion is this: if you need a fix, and a Twisted developer *doesn't* think it's important, the inclusion of unit tests will likely see it merged regardless.
(...) I would have written unit tests if someone told me to do it. (...)
Since you had to repeat yourself, let me repeat the apology. I was confused by the context of your objection.
I look at the speed of fixing bugs. And in this particular case it took 18 months or so to fix it. I have added other bug reports, without getting any reaction.
If you'd like to mention them off-list I will have a look and we can discuss the priority they're at and what other possibly undisclosed requirements or requests for more information that there are. I think tests are important, that doesn't mean I think people should feel ignored by the Twisted team when they're submitting valuable bug reports.
IMHO, having working code without tests is better than having buggy code without tests (well, one might argue that it's the same...).
I think that the parenthetical comment there pretty much makes my argument for me :).
I admit that it's nice for the twisted developers to have bug reports in the form of failing unit tests. But you can't expect everyone to write them.
Yes. Let me reiterate - if you discover a serious bug, report it, and it will be fixed, roughly in order of its importance (as determined by some random person on the Twisted project, whose priorities may not be in line with your own). You don't need to include any tests, or a patch. *If you have written a patch*, which is seriously important to you, and you want to do what you can to get it merged faster, copious unit tests, and a distant second, a good explanation of what it's for, will get it merged faster. If you can explain that it is covered by existing tests, and fixes an intermittent failure, that's almost as good. (For example, I added process support to the default reactor on Windows recently, and there were existing tests which covered that functionality that were previously disabled. I just turned them on.) If a Twisted developer chooses to fix your bug, it is almost certain that they will write their *own* tests when they do it, or get endless shit from the rest of us about it :).
I'm not whining (actually I write unit tests at work). We use twisted at our company and appreciate the work that has gone into it. But one thinks about saving that work of reporting bugs, when they just aren't dealt with.
It sounds like we agree on more of this than we disagree on, so I suspect that as I misread your message, you also slightly misread my reply. People should report bugs, people should send patches. They won't be harrassed for either, tests or no tests, nor will the patches and bugs be disregarded. They just shouldn't expect patches with no tests that are not directly in line with a developer's interest to be considered high-priority. (Of course, on a project with the developer/work ratio of Twisted, anything that isn't high-priority just isn't going to happen.)
On Fri, Jan 20, 2006 at 07:36:21PM +0100, Andrea Arcangeli wrote:
My test suite is the CPUShare server code, I tested it. This is a trivial patch and I hope I can be applied right away without special requirements (a bug was checked into trunk and this fixes it, plus it adds one more class that nobody can be using yet, so it can't make things worse). Or at least I've not nearly the time to write testsuites for such trivial fixes, nor to attach them somewhere with a web browser, sorry. I hoped I could contribute despite significant constraints, I'm sorry if that's not the case. I think development of twisted is too slow if it requires these formalities (especially given this is a web2 _unstable_ branch, I didn't touch anything else in my patch).
Unstable doesn't mean broken. Writing unittests might be boring but it's absolutely required for projects developed by more than 2 people. And the linux kernel shows how this is true quite well, and I also understand why unstable is equal to broken for you. Not counting the "many" rewrites that happen in the kernel each release. We don't have 200-to-1000 developers regularly payed by international corporations. Considering the free time we have and how many things are in twisted (and I've yet to see a project in any language being as complete as it is) I could say that the development is even fast. Nobody has always committed unittests together with fixes but still this is wrong and it's not yet another reason to insult the others' work especially when what you say is false. The rest of the post is pure FUD and trolling so I won't consider it.
Cheetah and FormKit). If something goes wrong in this process I'll drop twisted entirely from the server side since I've wasted enough time already (I can still use it on the client side since it worked well there, there's no http involved in the client side after all and furthermore there are no smp scalability concerns on the client side).
I'm sorry to say this but... given your latest behavior I'd be happy if you stopped using twisted at all since you are actually harming the community instead of making it better. Maybe more time spent on unittests, bug trackers and documentation or source code could have done better than the last 10-15 emails. I'm sure that the ever crashing cherrypy web server will serve your pages better and faster until it will crash for deadlocks. <g> -- Valentino Volonghi aka Dialtone Now Running MacOSX 10.4 Blog: http://vvolonghi.blogspot.com New Pet: http://www.stiq.it
On Fri, Jan 20, 2006 at 09:58:42PM +0100, Valentino Volonghi wrote:
I'm sorry to say this but... given your latest behavior I'd be happy if you stopped using twisted at all since you are actually harming the community instead of making it better. Maybe more time spent on unittests, bug trackers and documentation or source code could have done better than the last 10-15 emails.
If stuff like is checked into SVN trunk (instead of the file logging feature I posted!), it means I'm not the only one that could spend his time better. http://svn.twistedmatrix.com/cvs/trunk/doc/fun/Twisted.Quotes?r1=15551&r2=15652&p1=trunk/doc/fun/Twisted.Quotes&p2=trunk/doc/fun/Twisted.Quotes
On Sat, 21 Jan 2006 00:18:38 +0100, Andrea Arcangeli
If stuff like is checked into SVN trunk (instead of the file logging feature I posted!), it means I'm not the only one that could spend his time better.
(... Twisted.Quotes ...)
Let me clarify my position - unit tests are not required for commits to Twisted.Quotes. If you have a patch to that file we will evaluate it purely on its humorousness and relationship to the various individuals who contribute to Twisted on a regular basis. I apologize for not having previously stated the official requirements for inclusion in the quote file in a public forum outside of IRC. Seriously though, are you *trying* to convince us all that you are the biggest jerk in the world? First, you tell Divmod what we are and aren't allowed to do with our own code. Then, you waste a bunch of our time because you won't allow anyone to tell *you* what the requirements are for getting code included in Twisted. Finally, you tell the Twisted team what we can and can't do with our free time. Perhaps you have become confused and you believe you are in a managerial position because you, or CPUShare, have recently contributed a large sum of money to the Twisted team. Let me assure you that you that you are not - I just checked my paypal account, and it has nothing to say about you. Wasting our time and pissing us off with repetitive posting of (mostly incorrect) opinions to a mailing list is not quite the same thing as the Twisted developers getting together to have a good time on IRC. If you want to spend your time chatting with *your* friends on IRC (or even on AIM or MSN! Twisted supports all these protocols!) please, be my guest.
They say it's better to say nothing and be thought a fool. Allow me to demonstrate as I recklessly jump into an already overheated discussion. On Fri January 20 2006 18:39, glyph@divmod.com wrote:
Seriously though, are you *trying* to convince us all that you are the biggest jerk in the world?
It's been interesting following the list for the last week or so. Andrea's served, consciously or not, as a pretty fair Devil's Advocate. I've always looked forward to what the movers and shakers on the list will make out of the latest bug up his... bonnet. It's a credit to the list that things have remained as civil as they have, and that his concerns have been given as much attention as they have. But it's starting to get old. There's got to come a time when one has to just laugh, tussle his hair, and say, "Oh, that Andrea." Surely that time has come by now. For instance, and do correct me if I'm wrong, I think he's admitted in passing to not having used unit tests before. Like everything else that has come up which he isn't familiar with, he later dismissed them as pointless. How long can this sort of "that's not the way Mom does it" criticism be taken seriously? Andrea seems determined to find an answer to that question. Mike.
On Fri, Jan 20, 2006 at 07:05:48PM -0500, Mike Pelletier wrote:
admitted in passing to not having used unit tests before. Like everything
I've not used unit test in my CPUShare work related to Twisted, but that's because I've no time for that and this isn't going to change. Please don't confuse the kernel work I do, with the CPUShare work. The CPUShare work I do happens in the spare time with very little resources, so I'm not even dreaming of writing unit tests. In the kernel we use things like LTP or other stress test like cerberus to check for regression. But those packages are separated. In my 8 years of lkml experience I've never heard one single time that one fix or feature wasn't applied because the author didn't write a corresponding LTP testcase yet (I've heard all sort of arguments but not yet the missing unittest). The group of people doing LTP is infact almost not overlapping with the people writing the kernel code. Furthermore the LTP and the other stress tests are in userland, so there's a moltitude of kernel-internal apis they can't hope to test. However we've some subsystem module like the packet injection engine or the RCU torture test module that can stress some kernel internal apis to spot race conditions or bugs. But to make an example, the RCU torture test was posted and included _years_ after RCU made into the kernel and after being used by routing cache, IPC and dcache for years. The idea is to first make things working, then once stuff works, we can spend some time for the unittest. It's shoking for me to hear that valid bugfixes and features are left hanging around for years in bugrackers because no unit test exists. The twisted branch that CPUShare will use in production on the server side from now on is published at the below URL. This is a fork in the sense that I'll change whatever I need on the server side (while still providing backwards compatibility). I'll closely track twisted SVN trunk on a weekly or monthly (or daily) basis. I'll try to keep the thing working stable as much as I can, since it'll be used on live servers. For example if I feel twisted.web2 is changing too fast I may freeze it for a while in my branch to reduce the number of breakages to the apps. Fixes and new features will be applied promptly and without formalities. I'll send fixes upstream through the mailing list but I don't expect them to be applied due the lack of unittest. http://www.cpushare.com/hg/Twisted You can check it out with mercurial: http://www.selenic.com/mercurial/ http://www.selenic.com/mercurial/wiki/index.cgi/QuickStart quickstart: wget http://www.selenic.com/mercurial/mercurial-snapshot.tar.gz tar xzf mercurial-snapshot.tar.gz cd mercurial-snapshot python setup.py install hg clone http://www.cpushare.com/hg/Twisted /whereveryouwant/Twisted then edit /whereveryouwant/Twisted, "hg ci", and publish on your server so I can pull it back. here the twisted.web2 module to publish on your server (hgwebdir.cgi is from twisted dir): from twisted.web2 import server, twcgi, channel from twisted.internet import defer site = server.Site(twcgi.FilteredScript('hgwebdir.cgi', filters=['python'])) from twisted.application import service, strports application = service.Application("mercurial") s = strports.service('tcp:8080', channel.HTTPFactory(site)) s.setServiceParent(application)
How long can this sort of "that's not the way Mom does it" criticism be taken seriously?
Exactly. It looks like a typically Italian, self-centered, "Mommist" world view, and rather limiting too.
Content-Type: text/plain; charset="iso-8859-6"
Mmh... Arabic?!? -- Nicola Larosa - http://www.tekNico.net/ You asked about my "path to world domination". Well, I guess I want the whole world to be like the Oberlin CS Lab was: a sharing community where people assume that improving the system is within anyone's reach. -- Karl Fogel, November 2005
----- Original Message -----
From: "Nicola Larosa"
How long can this sort of "that's not the way Mom does it" criticism be taken seriously?
Exactly. It looks like a typically Italian, self-centered, "Mommist" world view, and rather limiting too.
i hate to break it to you, but behavioural generalizations based on nationality have rarely worked based on my observations. for example, i am neither an avid vodka drinker (no time to be wasted) nor a mafia operative (i wasn't good enough at battering people to death with blunt objects when i was a teenager).
Content-Type: text/plain; charset="iso-8859-6"
Mmh... Arabic?!?
... and what does that have to do with anything? i'm not the political correctness police by any means, but i fail to see how such remarks are relevant to this discussion or this list. i fear the 'not too long before godwin strikes' prediction has been more spot on than i'd hoped. -p
On Sat January 21 2006 03:54, Nicola Larosa wrote:
How long can this sort of "that's not the way Mom does it" criticism be taken seriously?
Exactly. It looks like a typically Italian...
Yikes. I was being rude, yeah, but I didn't mean anything like that. Apologies to Andrea and the list if I seemed to be implying it.
Content-Type: text/plain; charset="iso-8859-6"
Mmh... Arabic?!?
Don't ask me. KMail automatically chose the encoding based on the content. Strange choice, but not incorrect. Mike.
I've expressed my admiration for the Twisted project and the brilliance of its developers on numerous occassions. I like to think that my background in patenting things gives me an eye for spotting innovative work, and there's a *lot* of innovation going on with Twisted that, thankfully, Glyph et al. have essentially given to the public for its free use. I hope that background gives me a little room to concur in some of the criticism being aired here. Perhaps we are long overdue for the kind of hearing that Andrea is creating with his willingness to speak his mind. The Twisted team is great about sticking to good programming practices and showing professionalism and maturity in certain cases but comes across far differently in many others. The religious fervor around unit tests is a positive example, but the goofy naming conventions (jelly, banana, manhole, etc.) that don't impart much if any information to the source code reader are a very contrary case in point. Andrea points out another example below; there's a lot of just plain crud and useless amusements littering up the SVN tree, yet many classes and methods in the code are devoid of meaningful docstrings. I once made the mistake of trying to check out SVN/branches -rHEAD and found that there are several *gigabytes* of utterly worthless cruft in there, the vast majority of which will never, ever be touched. A huge debate on IRC ensued about the wisdom of keeping abandoned stuff in the HEAD revision of *any* SVN repository. You can guess what my viewpoint was, and I figured it fit well into the "do everything excellently even to the point of pain" vision of the Twisted developers, but I was wrong about that. I even saw a bit of this contradictory zeitgeist on the #twisted IRC channel recently, when someone was criticized for off topic posts about more general Python stuff. I pointed out that there is a lot of utter nonsense on the IRC channel, including a lot of frankly silly, repetitive stuff about poking, eating shoulders, stabbing in the eye, etc. Sure, this is all free software and we're all largely in it to have some fun, but I think there's some reasonable grounds for not-always positive comments from the user base, and that hardly is "harming the community." Best regards, Ed Andrea Arcangeli wrote:
On Fri, Jan 20, 2006 at 09:58:42PM +0100, Valentino Volonghi wrote:
I'm sorry to say this but... given your latest behavior I'd be happy if you stopped using twisted at all since you are actually harming the community instead of making it better. Maybe more time spent on unittests, bug trackers and documentation or source code could have done better than the last 10-15 emails.
If stuff like is checked into SVN trunk (instead of the file logging feature I posted!), it means I'm not the only one that could spend his time better.
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
On Jan 20, 2006, at 6:46 PM, Ed Suominen wrote:
Sure, this is all free software and we're all largely in it to have some fun, but I think there's some reasonable grounds for not-always positive comments from the user base, and that hardly is "harming the community."
I completely agree with that, in principle, but I think we've passed the reasonableness threshold on this set of threads at least 5 days ago. At the rate this discussion is degenerating, it shouldn't be long too long until the truth of Godwin's Law is demonstrated. James
On Fri, 20 Jan 2006 15:46:36 -0800, Ed Suominen
I've expressed my admiration for the Twisted project and the brilliance of its developers on numerous occassions. I like to think that my background in patenting things gives me an eye for spotting innovative work, and there's a *lot* of innovation going on with Twisted that, thankfully, Glyph et al. have essentially given to the public for its free use.
Hint to other people with criticism - this is a good way to start a message :) (Thanks, Ed)
I hope that background gives me a little room to concur in some of the criticism being aired here. Perhaps we are long overdue for the kind of hearing that Andrea is creating with his willingness to speak his mind.
Yes. I am frequently looking for criticism of various Twisted (and Divmod) projects; especially from competent people with specific goals in mind. I have been heard to complain about this, actually - useful criticism is hard to find. I probably hear more baseless hagiography than baseless insults. It's equally useless :). A pipe-dream of mine is to get a thorough efficiency review of the Twisted codebase from a really experienced C++ programmer; to have them just totally rip it apart and suggest (or better yet, of course, provide) a reactor implementation that is maximally efficient according to the best known practices for async I/O. Jason Beardsley, can you hear me? :) Perhaps we can turn this thread into a productive discussion.
The Twisted team is great about sticking to good programming practices and showing professionalism and maturity in certain cases but comes across far differently in many others. The religious fervor around unit tests is a positive example, but the goofy naming conventions (jelly, banana, manhole, etc.) that don't impart much if any information to the source code reader are a very contrary case in point.
There should *definitely* be more documentation around things like PB - and a bit less silliness in naming perhaps. However, I do note that the coding standard allows for silliness in module names, but forbids it in class, method, and function names. A certain amount of silliness is unavoidable. Corporate people call silliness "branding". What information does "ActiveX" or "Vista" or "Cocoa" impart to the user? but if you have any examples of where the standard has been violated, or you would like to improve the documentation of what the various silly names mean, *please* go ahead. (Although, probably PB doc bug discussions should go on the other mailing list...)
Andrea points out another example below; there's a lot of just plain crud and useless amusements littering up the SVN tree, yet many classes and methods in the code are devoid of meaningful docstrings.
Twisted.Quotes is a single file, and I have actually received quite a bit of positive feedback about the file's utility in understanding the IRC culture of the Twisted development team and lubricating social interactions that take place there, especially for people less familiar with IRC. It's amusing, and perhaps not as useful as code, but it's not "crud" or "useless".
I once made the mistake of trying to check out SVN/branches -rHEAD and found that there are several *gigabytes* of utterly worthless cruft in there, the vast majority of which will never, ever be touched. A huge debate on IRC ensued about the wisdom of keeping abandoned stuff in the HEAD revision of *any* SVN repository. You can guess what my viewpoint was, and I figured it fit well into the "do everything excellently even to the point of pain" vision of the Twisted developers, but I was wrong about that.
This, on the other hand, is definitely a useful observation. Let me try my hand at that productive-conversation thing I mentioned before! I missed this discussion, and if I hadn't, I would have agreed with you :-). /branches in the Twisted repository is insane. I actually removed some branches recently: around r15627. Not to be too hard on the developers responsible for it - much of the crap in there is auto-generated. cvs2svn generates branches if you *sneeze*. Some of it belongs to people who are hard to contact, and currently the rudeness of just deleting it without warning outweighs the benefits of having a smaller checkout of /branches. Divmod's repository tries hard to keep /branches to a reasonable size: everything at http://divmod.org/svn/Divmod/branches/ is something we're actually working on. You'll note there is nothing older than a week. I would like it *very much* if Twisted could get to that point.
I even saw a bit of this contradictory zeitgeist on the #twisted IRC channel recently, when someone was criticized for off topic posts about more general Python stuff. I pointed out that there is a lot of utter nonsense on the IRC channel, including a lot of frankly silly, repetitive stuff about poking, eating shoulders, stabbing in the eye, etc.
I think I can half-defend this behavior. The silliness is done by people who are active contributors. In reality, they're just people who were formerly active contributors or who are friends of contributors, which is why I can only half-defend it. The silliness is energizing to developers, as are on-topic questions. However, new people who show up to ask off-topic but technical questions are soaking up the time that #twisted's denizens want to spend either being silly or working on Twisted. If someone were to pay a Twisted developer to support Python (or some Linux distribution, as those questions are sometimes heard too) I'm sure that you'd find them very prompt and courteous, as well as knowledgeable. In many cases, this support fatigue is related to half a decade spent in #python, answering the same question hundreds or thousands of times. There's nothing wrong with simple Python questions, but there is an appropriate forum on IRC already, and there are plenty of people there who can help you.
Sure, this is all free software and we're all largely in it to have some fun, but I think there's some reasonable grounds for not-always positive comments from the user base, and that hardly is "harming the community."
The harm to the community comes from the frequent implications of things like "Divmod told me to use Atop, and it ruined my business!". Mr. Arcangeli hasn't ever stated that directly, of course, but he continues to strongly imply it. I consider this *personally* very insulting, since Divmod has put a lot of money, time, and energy (argubaly more than was wise) into building a reputation with the open-source community. That reputation has been immensely valuable, since we have gotten lots of help from tireless contributors like Matt Goodall. Such a thing can be easily destroyed, and future contributors repelled, by a few misleading comments about the availability of support, or lack thereof, for our code. The harm comes from continuously repeating the same vague, baseless differences of opinion with Twisted developers. The fact that Andrea Arcangeli happens to prefer code that is in the library model rather than the framework model is not a serious criticism of Twisted. Nor is the fact that he doesn't like to close over arguments rather than have them passed to him from the framework. There are LOTS of problems with Nevow. To mention a few: livepage.LivePage should be converted into a backward-compatibility wrapper around athena.LivePage. The flatten() system can get stuck in a dead loop if you do apparently-innocuous things. Lots of things are undocumented. Fragments largely obviate the need for the evil 'ctx' and 'data' arguments on every renderer. These are all problems about which very concrete things can be said. Here are some more: http://divmod.org/trac/query?status=new&status=assigned&status=reopened&groupdesc=1&group=priority&component=Nevow&order=priority You might ask, "Why then, glyph, do you bother to respond? Shouldn't you be fixing those problems?" I do actually have a reason :-). Comments like the ones we're talking about are discouraging. There are lots of developers working on webby Twisted things that are frequently frustrated enough to throw their hands up and go play Crack Attack instead, just because of the volume of *legitimate* problems we have to deal with. Responding to emphasize that this rudeness isn't important feedback is important. It turns a long-winded screed about how worthless all the Nevow team's efforts have been into an amusing flamewar ;-). Still, I *would* rather be spending my time on more positive ways to keep them energized.
On Fri, Jan 20, 2006 at 07:51:59PM -0500, glyph@divmod.com wrote:
The harm comes from continuously repeating the same vague, baseless differences of opinion with Twisted developers. The fact that Andrea Arcangeli happens to prefer code that is in the library model rather than the framework model is not a serious criticism of Twisted. Nor is the fact
I repeated it so many times but it seems you didn't got what I meant yet. It was never a criticism to Twisted, I was talking about Nevow. The only criticism I have to Twisted is that it doesn't scale in SMP, despite it theoretically could but nobody attempted it yet (ok, python would be bad at running it so I guess it wouldn't make much sense in practice even if it's an attractive idea in theory). Not scaling in smp is bad but I don't want to get into those details, for now getting twisted.web2 stable single threaded would be enough... The API provided by Twisted is not a limitation because it offers all features already, it's just an extension of the socket API (which is fixed in stone anyway). Twisted just provides an abstraction of it and that's good in my opinion. To make a parallel in the kernel we've lots of apis where you've to register into. The VFS is the most obvious example of that. Filesystems have to be written following a strict API and there's no (sane) way to make a filesystem without following those strict rules and those strict callbacks. Because again this API is the one that userland sees (reiser4 tried to break that rule, but I think they backed it out since it wasn't backwards compatible). However in a moltitude of other cases it's possible to create a library that is being called (instead of calling). USB is an example of that. The lowlevel drivers just use a library. It's the driver code that calls into the usb highlevel, not the other way around. parport is similar. When it's possible to do that, it's much nicer to do that, since it leaves an higher degree of flexibility to the developer. SCSI unfortunately didn't do that, and scsi become hard to maintain in fact. All scsi drivers in linux are just implementing callbacks and that's sometime not flexible enough (I had myself troubles with recursion into the scsi higlevel code when I was hacking on ppa initially). Please don't confuse my criticism to the nevow way of hooking inside the app way beyond its templating library job, with a criticism to twisted.
glyph@divmod.com writes:
Twisted.Quotes is a single file, and I have actually received quite a bit of positive feedback about the file's utility in understanding the IRC culture of the Twisted development team and lubricating social interactions that take place there, especially for people less familiar with IRC. It's amusing, and perhaps not as useful as code, but it's not "crud" or "useless".
Twisted.Quotes is *great* and the idea that time spent maintaining that file is time wasted that should have been spent, e.g., working on twisted.web2.client is so wrong-headed as to be completely baffling. Cheers, mwh -- Hmm, that attempt at trolling did not work out as well as I had hoped. -- Steve VanDevender, asr
participants (11)
-
Andrea Arcangeli
-
Ed Suominen
-
glyph@divmod.com
-
James Y Knight
-
Jean-Paul Calderone
-
Michael Hudson
-
Mike Pelletier
-
Nicola Larosa
-
Paul G
-
Ralf Schmitt
-
Valentino Volonghi aka Dialtone