I haven't looked at the patches, but couldn't we have the IPv6 code be #ifdef'ed out, so that those who care about IPv6 can periodically test it while the various OS-level libraries are ramped up over the next months/years, but w/o disturbing the 'current' builds?
Not if we are going to introduce itojun's patch. In that patch, the IPv6 code *is* actually ifdef'ed out. It is getaddrinfo/getnameinfo that gives problems, which isn't IPv6 specific at all. The problem is that the library patches (httplib, ftplib, etc) do use getaddrinfo to find out how to contact a remote system, which is the right thing to do IMO. So even if the IPv6 support can be activated only if desired, getaddrinfo absolutely has to work. So the only question then is where we get an implementation of these functions if the system doesn't provide one. itojun has suggested the WIDE libraries; since they apparently don't compile on Windows, I've suggested the MS TP emulation. If the latter is not acceptable, we either have to fix the WIDE implementation to work on Windows also; As for the problems Mark reported: I think they can get fixed. Regards, Martin
On Sun, Jun 24, 2001 at 07:48:03PM +0200, Martin v. Loewis wrote:
The problem is that the library patches (httplib, ftplib, etc) do use getaddrinfo to find out how to contact a remote system, which is the right thing to do IMO. So even if the IPv6 support can be activated only if desired, getaddrinfo absolutely has to work.
Why ? Why can't those parts be 'if it exists'-ed out ? We do it for SSL support. I'm only comfortable with the IPv6 patch if it's optional, or can at least be disabled. I haven't looked at the patch, but why is getaddrinfo absolutely necessary, if the code works without it now, too ?
So the only question then is where we get an implementation of these functions if the system doesn't provide one. itojun has suggested the WIDE libraries; since they apparently don't compile on Windows, I've suggested the MS TP emulation. If the latter is not acceptable, we either have to fix the WIDE implementation to work on Windows also;
As for the problems Mark reported: I think they can get fixed.
What about the zillion other 'obscure' ports ? OS/2 ? Palm ? MacOS 9 ;) If
this patch can't be zero-impact-if-necessary, I'm a firm -1 on it. But I
don't think it can't, it just takes more work.
--
Thomas Wouters
Why ? Why can't those parts be 'if it exists'-ed out ? We do it for SSL support. I'm only comfortable with the IPv6 patch if it's optional, or can at least be disabled. I haven't looked at the patch, but why is getaddrinfo absolutely necessary, if the code works without it now, too ?
getaddrinfo offers protocol-independent address lookup. It is necessary to use that API to support AF_INET and AF_INET6 transparently in application code. itojun proposes to change a number of standard library modules. Please have a look at the actual patch for details; the typical change will look like this (for httplib) diff -u -r1.35 httplib.py --- Lib/httplib.py 2001/06/01 16:25:38 1.35 +++ Lib/httplib.py 2001/06/24 04:41:48 @@ -357,10 +357,22 @@ def connect(self): """Connect to the host and port specified in __init__.""" - self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - if self.debuglevel > 0: - print "connect: (%s, %s)" % (self.host, self.port) - self.sock.connect((self.host, self.port)) + for res in socket.getaddrinfo(self.host, self.port, 0, socket.SOCK_STREAM): + af, socktype, proto, canonname, sa = res + try: + self.sock = socket.socket(af, socktype, proto) + if self.debuglevel > 0: + print "connect: (%s, %s)" % (self.host, self.port) + self.sock.connect(sa) + except socket.error, msg: + if self.debuglevel > 0: + print 'connect fail:', (self.host, self.port) + self.sock.close() + self.sock = None + continue + break + if not self.sock: + raise socket.error, msg def close(self): """Close the connection to the HTTP server.""" As you can see, the modified code can simultaneously access both IPv4 and IPv6 hosts, and will pick whatever it can connect to best. Without getaddrinfo, httplib would continue to support IPv4 hosts only. The IPv6 support itself is absolutely optional. If it is not available, getaddrinfo will never return IPv6 addresses, or propose AF_INET6 as the address family.
What about the zillion other 'obscure' ports ? OS/2 ? Palm ? MacOS 9 ;) If this patch can't be zero-impact-if-necessary, I'm a firm -1 on it. But I don't think it can't, it just takes more work.
Depends on what zero-impact-if-necessary means to you. The patch, as it stands, can be fixed to compile on all systems that are currently supported. It cannot be fixed to be taken completely out (unless you literally do that: take it out). I don't plan to fight for it too much. Please have a look at the code itself, and try to cooperate on integrating it. Don't reject it outright without having even looked at it. If I get strong rejections from everybody, I'll just withdraw it and feel sorry for the time I've already spent with it. Regards, Martin
martin wrote:
getaddrinfo offers protocol-independent address lookup. It is necessary to use that API to support AF_INET and AF_INET6 transparently in application code. itojun proposes to change a number of standard library modules. Please have a look at the actual patch for details; the typical change will look like this (for httplib)
diff -u -r1.35 httplib.py --- Lib/httplib.py 2001/06/01 16:25:38 1.35 +++ Lib/httplib.py 2001/06/24 04:41:48 @@ -357,10 +357,22 @@
def connect(self): """Connect to the host and port specified in __init__.""" - self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - if self.debuglevel > 0: - print "connect: (%s, %s)" % (self.host, self.port) - self.sock.connect((self.host, self.port)) + for res in socket.getaddrinfo(self.host, self.port, 0, socket.SOCK_STREAM): + af, socktype, proto, canonname, sa = res + try: + self.sock = socket.socket(af, socktype, proto) + if self.debuglevel > 0: + print "connect: (%s, %s)" % (self.host, self.port) + self.sock.connect(sa) + except socket.error, msg: + if self.debuglevel > 0: + print 'connect fail:', (self.host, self.port) + self.sock.close() + self.sock = None + continue + break + if not self.sock: + raise socket.error, msg
instead of adding code like that to every single module, maybe we should add a convenience function to the socket module? (and make that function smart enough to work also if getaddrinfo isn't supported by the native platform...) </F>
[Martin v. Loewis]
... So the only question then is where we get an implementation of these functions if the system doesn't provide one. itojun has suggested the WIDE libraries; since they apparently don't compile on Windows, I've suggested the MS TP emulation. If the latter is not acceptable, we either have to fix the WIDE implementation to work on Windows also;
I don't have cycles for this, but will cheerily <wink> suggest that the WIDE problems didn't appear especially deep, just "the usual" careless brand of Unix+gcc+glibc specific coding. For example, HAVE_LONG_LONG is #define'd on Windows, but, just as in Python source, you can't *use* "long long" literally, you have to use the LONG_LONG macro instead. Then Windows doesn't have an offsetof() macro, or an snprintf() either. Etc. The code is in trouble exactly where it relies on platform-specific extensions to the std C language and library. Problems with those won't be unique to Windows, either, which is a deeper concern (but already well expressed by others). It would be nice if Python could contribue portability back to WIDE. That requires worker bees, though, and lots of x-platform testing. If it turns out we can't swing that, then support for this is premature, and we should wait, e.g., for WIDE to put more effort into porting their code.
The problem is that the library patches (httplib, ftplib, etc) do use getaddrinfo to find out how to contact a remote system, which is the right thing to do IMO. So even if the IPv6 support can be activated only if desired, getaddrinfo absolutely has to work.
Yes, but in an IPv4-only environment it would be super trivial to implement, right? --Guido van Rossum (home page: http://www.python.org/~guido/)
The problem is that the library patches (httplib, ftplib, etc) do use getaddrinfo to find out how to contact a remote system, which is the right thing to do IMO. So even if the IPv6 support can be activated only if desired, getaddrinfo absolutely has to work.
Yes, but in an IPv4-only environment it would be super trivial to implement, right?
Right, and getaddrinfo.c/getnameinfo.c attempt such an implementation. They might attempt to get it "more right" than necessary, but still they are "pure C", in the sense that they don't rely on any libraries except for those available in a typical IPv4 sockets implementation. At least that's the theory. It turns out that they've been using inet_pton and snprintf, which is probably because they have been mainly tested on BSD. I'm in good faith that we can reduce them to a "no funny library calls needed" minimum. If somebody wants to implement them anew from ground up, only using what the socketmodule already uses, that would be fine as well. An actual review for the code for portability problems would also be helpful. Regards, Martin
participants (5)
-
Fredrik Lundh -
Guido van Rossum -
Martin v. Loewis -
Thomas Wouters -
Tim Peters