From: Ivo van der Wijk Subject: Re: [Twisted-Python] Proposal: changes to reactor.clientTCP Date: Sun, 16 Jun 2002 01:06:51 +0200

On Sat, Jun 15, 2002 at 06:04:29PM +0200, Itamar Shtull-Trauring wrote:

...

Background: Ivo pointed out there's no way to differentiate between "no such hostname" and "connection refused". For that matter, connectionFailed has nothing to do with the protocol - why are we doing a callback on the protocol at all? It shouldn't be created if no connection was made.

Proposal: reactor.clientTCP accepts a protocol factory instead of a protocol, and only makes a protocol instance if connection was made. Additionally, it accepts an additional argument, failureCallback, which will get called with a status code (CONNECTION_REFUSED, NO_SUCH_HOSTNAME) or something when the connection fails. Protocol.connectionFailed goes bye-bye.

You know, I'm tempted to disagree with this because backwards-compatibility at this point is a really major argument to me at this point ... however, you guys make some really good arguments. It may even be possible, given this description, to make __init__, and not makeConnection, be the real "initializer" of a Protocol at this point. The inability to report these errors has also been an annoyance on the client-interface end of things for some time. There is an asymmetry between client and server-side connecting in IReactorTCP. In addition, Itamar's proposal would simultaneously eliminate the confusion currently surrounding Connectors, too; having a wrapper protocol *factory* would not pose the same efficiency concerns as a wrapper Protocol instance, since indirection would only happen when opening and closing the connection.

...

Comments?

Some people don't see a point in providing these errors, so I'll illustrate the issue with some more examples:

Also, being able to report these errors is important to judge at what level networking is misconfigured for the end-user, when they can't connect.

I agree with itamar that if these errors aren't handled in the protocol implementation, failure to connect shouldn't be there either.

After hearing this, so do I. Thank you for the bug report. -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |

From: Glyph Lefkowitz Subject: Re: [Twisted-Python] Proposal: changes to reactor.clientTCP Date: Tue, 18 Jun 2002 19:38:46 -0500 (CDT)

From: Tommi Virtanen Subject: Re: [Twisted-Python] Proposal: changes to reactor.clientTCP Date: 18 Jun 2002 21:12:52 +0300

...

Please do not clobber the information given by the kernel in errno. Translating errno to 2 different failures is not nice.

It needs to be clobbered, at least in some sense, for portability.

Jython does not have errno and win32's support for it is very weak.

Reading this a few hours later, I think my tone was wrong and I was altogether too brief here. :-). To rephrase in a more constructive manner: I believe that it needs to be clobbered, and some information lost, because I don't see any other way to retain portability. Also, we're losing information already, since errno is used to report things that aren't "errors" in the Twisted worldview (or at least, not errors that the user can deal with directly). Jython and win32 both have error-reporting mechanisms; Jython's is exceptions, win32 is a sort of wacked-out informal errno. Can you think of a few primitive errors that we should support, rather than just the 2 types? (I'd rather use Failure than just integer error codes, personally.) I think that if we're going to add a new error reporting mechanism we need to do it carefully so that we don't miss the same kind of information that we did the last time we architected this API. (Protocol.connectionLost used to take an integer error code which was one of these two errors; that was a bad idea, too). Thanks for your feedback. Does anybody have a solution? -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |

From: Tommi Virtanen Subject: Re: [Twisted-Python] Proposal: changes to reactor.clientTCP Date: 21 Jun 2002 22:04:08 +0300

Allow portable programs to get a "sanitized version", but allow my POSIX-only programs get errno. For what kind of sanitization one might want, you probably want to look at the man page for say connect(2) for all the possible error cases, and categorize them based on local/remote, temporary/permanent, etc., and create a "high-level error" for each class, with perhaps the string corresponding to the errno as extra info meant for users (and not if statements).

But please do _not_ loose any information in the translation. Say people shouldn't look at the error objects errno attribute to decide what to do, as not all platforms provide it. But don't hide the actual error.

Upon consideration I strongly agree that no information should be lost. So, let's have such a high-level error, and a .platformError attribute on the connection error object which is an object specific to the "OS" that you're running under. On POSIX this will be (or contain) an errno error code, but on Jython and elsewhere it will be whatever's appropriate. Does that work for you?

Remember my background; I'm a kernel hacker, and I know the UNIX libc/kernel API. If I run a program that tries to e.g. connect with TCP, and it fails without giving me a good reason, I _will_ run it under strace -e trace=connect just to see whether connect failed, and with what errno. Operating systems without BSD sockets just aren't, and I write my programs to use fork anyway. My point -- don't decide for me, let me decide on my own.

The reason I'm resistent to this is that I would like to encourage writing portable programs with Twisted, especially in cases where platform specificity doesn't add any functionality. On the other hand, I don't want to be Java here and try to _force_ programmers to write their programmers in a portable way when they really know better than I do what they want. Problem is, programmers rarely know what they want. For example; if you're using Twisted, why use fork? You ought to be using reactor.spawnProcess for non-portability-related reasons anyway; Python's refcounting doesn't work reliably with COW memory. Of course, there are reasons not to do this; of course, there are times when you just _need_ to get down to the OS and do something in a very, very platform specific way (sometimes even specific to your particular OS vendor, not just POSIX). This should not be the default. Handling an errno error code specifically should also be considered a *very* esoteric operation. I personally can't come up with any need to use it except debugging under a particular environment; I would far rather that people used to the POSIX API learned how to use the Twisted APIs to do POSIX-y things, so that their programs would be more portable. Balancing this with the need not to piss people off is difficult :-). I can't think of a way to say "DON'T USE .platformError UNLESS YOU REALLY NEED IT, WHICH YOU DON'T" without making everybody read the documentation, which they won't anyway. I can imagine many programs made non-portable by someone looking for the answer to the question "how do I find the errno of what happened" rather than the correct question "how do I handle a connection error". So I guess I'll provide the platform specific information and put my faith in the worldview of applications programmers; please try to give them good examples to look at :) -- | <`'> | Glyph Lefkowitz: Traveling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |

Glyph Lefkowitz writes:

So, let's have such a high-level error, and a .platformError attribute on the connection error object which is an object specific to the "OS" that you're running under. On POSIX this will be (or contain) an errno error code, but on Jython and elsewhere it will be whatever's appropriate. Does that work for you?

Yes.

Balancing this with the need not to piss people off is difficult :-). I can't think of a way to say "DON'T USE .platformError UNLESS YOU REALLY NEED IT, WHICH YOU DON'T" without making everybody read the documentation, which they won't anyway. I can imagine many programs made non-portable by someone looking for the answer to the question "how do I find the errno of what happened" rather than the correct question "how do I handle a connection error".

Don't document platformError outside the source code, and there point out that it should not be used, and point to the real solution. This is a social problem, and technical methods will not be able to solve it. And, if you want me to write code using the high-level API, the high-level API needs to be able to tell me atleast 1) if the error is local or remote 2) if the error is temporary or permanent 3) how to report the error to the user without losing information -- tv@{{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} double a,b=4,c;main(){for(;++a<2e6;c-=(b=-b)/a++);printf("%f\n",c);}