Issues with Py3.1's new ipaddr
In http://bugs.python.org/issue3959 , Clay McClure is raising some objections to the new ipaddr.py module. JP Calderone shares his concerns. I think they were the only commenters not directly affiliated with one of the competing projects. The issues they've raised seem serious, but I don't know enough about the subject to make a meaningful comment. Am hoping python-dev participants can provide some informed judgments. At issue is whether the module has some deep conceptual flaws that would warrant pulling it out of the 3.1 release. Also at issue is whether the addition was too rushed (see David Moss's comment on the tracker and later comments by Antoine Pitrou). Does anyone here know if Clay's concern about subnets vs netmasks in accurate and whether it affects the usability of the module? Raymond
I haven't read the bug, but given the extensive real-life use that ipaddr.py has seen at Google before inclusion into the stdlib, I think "deep conceptual flaws" must be an overstatement. There may be real differences of opinion about the politically correct way to view subnets and netmasks, but I don't doubt that the module as it stands is usable enough to keep it in the stdlib. Nothing's perfect. I think we should just stick to "sorry, too late, try again for 3.2". We've done that with plenty of more important flaws that were discovered on the verge of a release, and I don't recall ever regretting it. We can always add more features to the module in 3.2. --Guido On Mon, Jun 1, 2009 at 11:32 AM, Raymond Hettinger <python@rcn.com> wrote:
In http://bugs.python.org/issue3959 , Clay McClure is raising some objections to the new ipaddr.py module. JP Calderone shares his concerns. I think they were the only commenters not directly affiliated with one of the competing projects. The issues they've raised seem serious, but I don't know enough about the subject to make a meaningful comment.
Am hoping python-dev participants can provide some informed judgments. At issue is whether the module has some deep conceptual flaws that would warrant pulling it out of the 3.1 release. Also at issue is whether the addition was too rushed (see David Moss's comment on the tracker and later comments by Antoine Pitrou).
Does anyone here know if Clay's concern about subnets vs netmasks in accurate and whether it affects the usability of the module?
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
On Mon, 1 Jun 2009 at 11:32, Raymond Hettinger wrote:
Does anyone here know if Clay's concern about subnets vs netmasks in accurate and whether it affects the usability of the module?
I can't speak to usability of the module, not having looked at it yet, but as far as I know from 10+ years of experience programming Cisco and other routers, a host address can always also be considered as a subnet consisting of one address. The netmask is 255.255.255.255, and is called a "host mask". That said, address+hostmask is not always the way you _think_ about an individual address; it depends on the context. FWIW I hate dealing with non-subnet-aligned IP address ranges whenever they come up. But it is true that they do come up. --David
Raymond Hettinger wrote:
Also at issue is whether the addition was too rushed (see David Moss's comment on the tracker and later comments by Antoine Pitrou).
That comment was from January 2009; it had two aspects a) is ipaddr getting special treatment just because it was contributed by Google? and b) shouldn't alternatives be considered? For a), the answer was always a clear "no". That the code comes from Google didn't affect inclusion at all. For b), there have been several attempts to approach alternatives since. For example, both netaddr people and ipaddr people tried to merge the projects, unsuccessfully. Both Duncan McGreggor and David Moss eventually spoke in favor of including ipaddr. So I think this particular issue was resolved. As for Clay McLure's issue: I feel it's primarily a matter of taste. I see nothing morally wrong in using the same class for hosts and networks, i.e. representing a host as a network of size 1. I can understand why people dislike that, but I don't see why it would stop people from doing with the library what they want to do. A number of other features might be missing from the library, for example, as Clay complains, there is no support for ranges of addresses (or, more generally, arbitrary sets). While I understand that there are good applications for such a data structure, I also think it can be added when contributed (and I would think "range of addresses" is still too narrow, for some applications I would envision - such as computing consistency of BGP tables and whois databases). Regards, Martin
On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. Löwis" <martin@v.loewis.de>wrote:
As for Clay McLure's issue: I feel it's primarily a matter of taste. I see nothing morally wrong in using the same class for hosts and networks, i.e. representing a host as a network of size 1. I can understand why people dislike that, but I don't see why it would stop people from doing with the library what they want to do.
To the extent that Clay is having issues, it's because ipaddr.py is poorly documented, has potentially confusing comments, and he became confused. Lesser issues are that ipaddr.py doesn't work the way he wants and that ip addressing is inherently subtle. Looking at the code in detail shows that ipaddr.IP/IPv4/IPv6 objects always represent *networks*. He wants one particular address out of that network, and that requires using __getitem__ or using IP.ip_ext. Neither is particularly intuitive.
import ipaddr ip = ipaddr.IPv4('10.33.11.17') ip IPv4('10.33.11.17/32') ip[0] '10.33.11.17' ip.ip_ext '10.33.11.17'
This feels much more like poor documentation than wide-ranging conceptual flaws. I could put this in the tracker, but I'm not sure if that's appropriate. -jake
On Mon, Jun 1, 2009 at 6:54 PM, Jake McGuire <mcguire@google.com> wrote:
On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. Löwis" <martin@v.loewis.de>wrote:
As for Clay McLure's issue: I feel it's primarily a matter of taste. I see nothing morally wrong in using the same class for hosts and networks, i.e. representing a host as a network of size 1. I can understand why people dislike that, but I don't see why it would stop people from doing with the library what they want to do.
To the extent that Clay is having issues, it's because ipaddr.py is poorly documented, has potentially confusing comments, and he became confused. Lesser issues are that ipaddr.py doesn't work the way he wants and that ip addressing is inherently subtle.
Sorry for the spam, I wrote my last message before reading the entire discussion surrounding the two libraries and trying to imagine using both ipaddr and netaddr. Clay is basically correct. The ipaddr.py API is missing important features, and it would probably be a mistake to add it to the python standard library if that means we'd have to maintain compatibility for the indefinite future. Like all largeish python projects, we wrote a library to do IP address manipulation. In our case, it's a whopping 64 lines long. While I wasn't aware of ipaddr.py or netaddr at the time, the API we created is much closer to netaddr's. Migrating our code to ipaddr would require significant work and is unlikely to happen. In fact, if I was starting a new project from scratch with similar requirements, I'd probably write my own library instead of using ipaddr. -jake
Clay is basically correct. The ipaddr.py API is missing important features, and it would probably be a mistake to add it to the python standard library if that means we'd have to maintain compatibility for the indefinite future.
From a maintenance point of view, these two statements don't really relate. Yes, adding it to the standard library means that compatibility must be maintained (not for the indefinite future, but in a very strong sense). But no, that doesn't mean that it cannot have new features.
Adding new features would not have to break compatibility, and, in many real-world cases of existing libraries, didn't. For the net-vs-host issue, I think a backwards-compatible solution is possible: just give the IP() function an option parameter that makes it reject a netmask during parsing.
Like all largeish python projects, we wrote a library to do IP address manipulation. In our case, it's a whopping 64 lines long. [...] In fact, if I was starting a new project from scratch with similar requirements, I'd probably write my own library instead of using ipaddr.
That was my feeling as well when ipaddr was first offered. It's just not an important library, and people will continue to roll their own for some time. OTOH, with ipaddr in the standard library, people will also start contributing extensions that make it support their use cases, so it should grow a wider application area than it currently supports. Regards, Martin
On Tue, Jun 2, 2009 at 1:38 AM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
For the net-vs-host issue, I think a backwards-compatible solution is possible: just give the IP() function an option parameter that makes it reject a netmask during parsing.
That doesn't solve much. IPv4 objects still always use CIDR notation when coerced to strings, meaning that IP addresses will always be rendered with a trailing "/32". Such notation is unacceptable in real-world applications that (correctly) distinguish between address and network.
That was my feeling as well when ipaddr was first offered. It's just not an important library, and people will continue to roll their own for some time. OTOH, with ipaddr in the standard library, people will also start contributing extensions that make it support their use cases, so it should grow a wider application area than it currently supports.
That being the case, why not delay its inclusion until we can be sure that it in fact represents a good base upon which to build? Clay
Clay McClure wrote:
On Tue, Jun 2, 2009 at 1:38 AM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
For the net-vs-host issue, I think a backwards-compatible solution is possible: just give the IP() function an option parameter that makes it reject a netmask during parsing.
That doesn't solve much. IPv4 objects still always use CIDR notation when coerced to strings, meaning that IP addresses will always be rendered with a trailing "/32".
That's not true: py> x = ipaddr.IP("30.40.50.60") py> print(x.ip_ext_full) 30.40.50.60
Such notation is unacceptable in real-world applications that (correctly) distinguish between address and network.
So use a different notation that is also supported by the library.
That was my feeling as well when ipaddr was first offered. It's just not an important library, and people will continue to roll their own for some time. OTOH, with ipaddr in the standard library, people will also start contributing extensions that make it support their use cases, so it should grow a wider application area than it currently supports.
That being the case, why not delay its inclusion until we can be sure that it in fact represents a good base upon which to build?
Because we *are* sure that it in fact represents a good base upon which to build. Regards, Martin
On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
That doesn't solve much. IPv4 objects still always use CIDR notation when coerced to strings, meaning that IP addresses will always be rendered with a trailing "/32".
That's not true:
py> x = ipaddr.IP("30.40.50.60") py> print(x.ip_ext_full) 30.40.50.60
Thankfully the authors have provided this obscure and strangely-named method to get at the correct string representation of an IP address, but sadly their __str__ method -- which is the Pythonic way to get string representations of objects -- fails in this regard because they have only one class representing two distinct concepts. I could probably make ipaddr work in my application; that is not the issue (at least in my mind). The issue is that I shouldn't have to work around design flaws in the library when they're simple enough to fix before the library is bundled with the stdlib. I do not see the utility of including ipaddr when my choice as a user is to either use its strangely-named methods instead of obvious Pythonic conventions, or to write my own library.
Such notation is unacceptable in real-world applications that (correctly) distinguish between address and network.
So use a different notation that is also supported by the library.
I'm not referring to my software here -- I'm referring to applications like ifconfig that expect addresses to be formatted properly. If the default string representation produced by the ipaddr library does not match the canonical representation expected by software that has existed as long as IP itself, that indicates to me the library is doing something wrong.
Because we *are* sure that it in fact represents a good base upon which to build.
You certainly seem convinced. I could not disagree more. Clay
On Tue, Jun 2, 2009 at 9:26 AM, Clay McClure <clay@daemons.net> wrote:
On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
That doesn't solve much. IPv4 objects still always use CIDR notation when coerced to strings, meaning that IP addresses will always be rendered with a trailing "/32".
That's not true:
py> x = ipaddr.IP("30.40.50.60") py> print(x.ip_ext_full) 30.40.50.60
Thankfully the authors have provided this obscure and strangely-named method to get at the correct string representation of an IP address, but sadly their __str__ method -- which is the Pythonic way to get string representations of objects -- fails in this regard because they have only one class representing two distinct concepts.
The minimal demonstration of the problem of representing networks and addresses using the same class:
container = [1, 2, 3, 4] for item in container: ... print "%s in %s: %s" % (item, container, item in container) ... 1 in [1, 2, 3, 4]: True 2 in [1, 2, 3, 4]: True 3 in [1, 2, 3, 4]: True 4 in [1, 2, 3, 4]: True import ipaddr container = ipaddr.IP('192.168.1.0/24') for item in container: ... print "%s in %s: %s" % (item, container, item in container) ... Traceback (most recent call last): File "<stdin>", line 2, in <module> File "ipaddr.py", line 438, in __contains__ return self.network <= other.ip and self.broadcast >= other.broadcast AttributeError: 'str' object has no attribute 'ip'
-jake
On Tue, Jun 2, 2009 at 1:11 PM, Jake McGuire <mcguire@google.com> wrote:
The minimal demonstration of the problem of representing networks and addresses using the same class:
fwiw, that (hosts vs networks in the same class) is not what you are demonstrating below. What you demonstrate is that its silly for iteration over a network to return strings rather than IP objects. Regardless, it is a good example of a problem with the API. ipaddr could be changed to yield IPv4 /32 or IPv6 /128 objects when iterating over it instead of strings and this example would work properly.
container = [1, 2, 3, 4] for item in container: ... print "%s in %s: %s" % (item, container, item in container) ... 1 in [1, 2, 3, 4]: True 2 in [1, 2, 3, 4]: True 3 in [1, 2, 3, 4]: True 4 in [1, 2, 3, 4]: True import ipaddr container = ipaddr.IP('192.168.1.0/24') for item in container: ... print "%s in %s: %s" % (item, container, item in container) ... Traceback (most recent call last): File "<stdin>", line 2, in <module> File "ipaddr.py", line 438, in __contains__ return self.network <= other.ip and self.broadcast >= other.broadcast AttributeError: 'str' object has no attribute 'ip'
Regardless, after reading the many different opinions on this thread especially including those of other python developers it sounds like we should not include ipaddr in 3.1. This example is IMHO one good reason to not include ipaddr in the standard library as it stands. It sounds like we have a 3.1rc2 scheduled for June 13th. At this point based on the multitude of other developer comments in these threads and barring strong arguments in favor of including it as it stands I will remove it because there seem to be quite a number of people with objections to it as an API in its current form. I've filed http://bugs.python.org/issue6182 as a release blocker to track its removal (or not) based on any further discussion in these python-dev threads. I do believe an API for an ip and network address manipulation library can be worked out but during the 3.1 release candidate phase is not the time to hastily do that and choose one so removal due to disagreement seems the best option. Would someone volunteer write up a proposal (PEP) for such with a goal of having it completed within the next few months? (I know people have suggested various other libraries, that counts; i have not taken the time to look at them). -gps
On Tue, Jun 2, 2009 at 10:41 PM, Gregory P. Smith <greg@krypto.org> wrote:
ipaddr could be changed to yield IPv4 /32 or IPv6 /128 objects when iterating over it instead of strings and this example would work properly.
I have opened an issue in the ipaddr bug tracker that I think addresses this issue. There are a few methods and properties in the ipaddr.IP classes that return native types but should return IP objects: http://code.google.com/p/ipaddr-py/issues/detail?id=21
It sounds like we have a 3.1rc2 scheduled for June 13th. At this point based on the multitude of other developer comments in these threads and barring strong arguments in favor of including it as it stands I will remove it because there seem to be quite a number of people with objections to it as an API in its current form.
Thank you.
Would someone volunteer write up a proposal (PEP) for such with a goal of having it completed within the next few months? (I know people have suggested various other libraries, that counts; i have not taken the time to look at them).
I am happy to take a stab at that. Thanks, Clay
On Tue, 2 Jun 2009 at 12:26, Clay McClure wrote:
On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. L�wis" <martin@v.loewis.de> wrote:
py> x = ipaddr.IP("30.40.50.60") py> print(x.ip_ext_full) 30.40.50.60
Thankfully the authors have provided this obscure and strangely-named method to get at the correct string representation of an IP address, but sadly their __str__ method -- which is the Pythonic way to get string representations of objects -- fails in this regard because they have only one class representing two distinct concepts.
Having thought more about this, I will agree with you that it would be useful to have an address-without-netmask class. I'm thinking about the cases where having an attached netmask is not particularly helpful, for example in DNS entries. But there are only two reasons I've so far come up with why this would be useful: to have a different default output format, and to have a way to encode an IP so that I can have my code raise an error if I try to use it in a context where a netmask is required. (Note, however, that even a DNS entry can conceptually be considered to be a host route.) IMO, not having such a class is an inconvenience, not a show stopper, especially since it seems like one could be added without breaking backward compatibility. I also don't particularly like the names of the ipaddr attributes for accessing the IP address-without-netmask or the netmask-without-ip-address, etc; but again, I don't consider that a show stopper. So I'm not in favor of pulling ipaddr from 3.1, and it's too late in the release cycle to change anything. I wish you had brought this energy to bear earlier, when changes could have been made. Reality is what it is, though, and now we should work on making improvements for the next release. I see in the ticket that the netaddr folks were going to propose improvements so that they could build netaddr on top of ipaddr, but I guess that didn't happen (yet?). I have no association with Google, by the way, and I do intend to use ipaddr in upcoming code, and have hacked my own address manipulation stuff in previous code.
Such notation is unacceptable in real-world applications that (correctly) distinguish between address and network.
So use a different notation that is also supported by the library.
I'm not referring to my software here -- I'm referring to applications like ifconfig that expect addresses to be formatted properly. If the default string representation produced by the ipaddr library does not match the canonical representation expected by software that has existed as long as IP itself, that indicates to me the library is doing something wrong.
I don't understand why you are trying to use ifconfig as an example. It is actually a counter example to your thesis: when working with an IP address intended for consumption by ifconfig, you had best be using a datatype that includes a netmask for that IP, or your code is going to be broken. What's more, modern versions of ifconfig _do_ accept the default output format of ipaddr. So this does the Right Thing: myip = ipaddr.IP('192.168.1.1/26') system('ifconfig eth0 {}'.format(myip)) If you use your ip-address-only class with ifconfig, you will wind up using the classful default netmask, which is only going to be correct by luck. Hmm. I think there is a conceptual divide here. You have said you think about IP addresses and networks as separate objects, so I wonder if you would be pulling the netmask for ifconfig out of a separate network object? On the other hand I, a network professional, think about an IP address paired with a netmask as a fundamental object. Very rarely do I use an IP address in isolation (without a netmask), and in many of _those_ cases there is an implied netmask of 255.255.255.255. Networks to me are closely related objects, defined by their "network number" (the zero of the subnet, which is an IP address and normally not used as a host address) and the mask. So to me ipaddr's use of a single datatype makes reasonable sense. I would, as I said, above, find an ip-without-netmask data type to be also useful (but a lot less important!) I would also find a subclass that was network-only (rejects anything but the zero of the subnet for the IP) useful. But I think both of those can be implemented fairly trivially as subclasses of the existing ipaddr objects. --David PS: I've looked briefly at netaddr, and while I could probably work with it by adding some equally trivial support code, I don't think it would serve my needs as well as ipaddr will. In particular, this is _very_ troubling:
import netaddr ip = netaddr.IP('192.168.1.1/26') ip IP('192.168.1.1/26') ip2 = netaddr.IP('192.168.1.1/27') ip2 IP('192.168.1.1/27') ip == ip2 True
The docs say the netmask is accepted only for "user convenience", but to me the netmask is an integral part of the data entity I want to manipulate. Nor can I express ip-address-with-netmask using the CIDR data type, since it will not accept anything but the zero of the network as the IP. In short, netaddr's object model does not match my desired model, while ipaddr's is a lot closer to my desired model.
On Tue, Jun 2, 2009 at 4:53 PM, R. David Murray <rdmurray@bitdance.com> wrote:
Having thought more about this, I will agree with you that it would be useful to have an address-without-netmask class.
Not only useful, but necessary. It seems there are few people on this list who understand IP well enough to realize how distorted ipaddr actually is.
(Note, however, that even a DNS entry can conceptually be considered to be a host route.)
Not at all. A host route and a host address (as represented in DNS) are fundamentally different concepts. Please see my recent post to ipaddr-py-dev for a refresher on these concepts: http://groups.google.com/group/ipaddr-py-dev/t/94d54fe581d24e72
So I'm not in favor of pulling ipaddr from 3.1, and it's too late in the release cycle to change anything.
I'm not sure why you say that when others have said that another release candidate is planned, and that removing ipaddr is essentially trivial to do.
I wish you had brought this energy to bear earlier, when changes could have been made. Reality is what it is, though, and now we should work on making improvements for the next release. I see in the ticket that the netaddr folks were going to propose improvements so that they could build netaddr on top of ipaddr, but I guess that didn't happen (yet?).
I don't think that can happen, actually. If I was a netaddr committer (which I'm not), I would find it hard to salvage anything reusable from ipaddr. It is certainly simpler and clearer to start over with an object model that actually makes sense.
I have no association with Google, by the way, and I do intend to use ipaddr in upcoming code, and have hacked my own address manipulation stuff in previous code.
Sorry, I wasn't aware of that. My mistake. Regardless, I find that your understanding of IP is similar to that of ipaddr's authors, which is to say imprecise.
I don't understand why you are trying to use ifconfig as an example.
Because it's an obvious real world example that explains why these two strings are not equivalent: 192.168.1.1 and 192.168.1.1/32 You and others continue to suggest that those strings are equivalent, yet ifconfig is a tool that has been around for thirty years that clearly demonstrates that those strings are not equivalent. If what you say is true, I should be able to pass either string to ifconfig and get the same result. That is not the case, because the strings are not equivalent, because a host route is not the same thing as a host address.
So this does the Right Thing:
myip = ipaddr.IP('192.168.1.1/26') system('ifconfig eth0 {}'.format(myip))
Sure, but shouldn't this also do the right thing? address = ipaddr.IP('192.168.1.1') netmask = ipaddr.IP('255.255.255.192') system("ifconfig eth0 %s/%s" % (address, netmask)) It doesn't.
Hmm. I think there is a conceptual divide here. You have said you think about IP addresses and networks as separate objects, so I wonder if you would be pulling the netmask for ifconfig out of a separate network object?
Of course, because addresses don't have masks; networks do. This command: ifconfig en0 192.168.1.1/24 is shorthand for operator convenience. What's going on behind the scenes is quite a lot different than it looks. First, ifconfig computes a network address by masking the supplied interface address with the supplied network mask. It is then able to configure a route for the proper network: 192.168.1.0/24. The fact that "192.168.1.1/24" appears in the command does *not* mean that the address 192.168.1.1 has a mask of /24. That is absurd. Addresses don't have masks; networks do. That's why they're called netmasks.
On the other hand I, a network professional, think about an IP address paired with a netmask as a fundamental object.
The IT industry is unique among engineering disciplines in that formal training and licensing are typically not required for IT professionals. Whereas concepts like resistance, current, and voltage have very specific meanings to electrical engineers, the IT vernacular is not so precise. Since formal training is rare, and what little is available is often high-priced and vendor-specific, IT professionals tend to learn their trade from trade books, word of mouth, and hands-on experience. As a result, IT professionals tend to have a good working knowledge of how technology applies to their particular job, but may not have an appreciation of the more theoretical aspects of the discipline. What this means in practice is that your experience as a network professional may not resemble the experiences of other network professionals. That you tend to think of addresses as having masks is probably not universal, or even particularly common, among network professionals. Some electrical engineers probably think of voltage as pressure, and that may be a useful abstraction, but I would be surprised to see a voltmeter calibrated in pascals. What are we to do? How do we arrive at a common understanding of our domain? We should consult the canonical sources of truth: RFC-791, and the BSD IP implementation. In neither of those will you see that an IP address has a mask. If this were any other problem domain, I think it should be obvious that the design of the library is flawed. But given that this is IP, a subject that many people think they understand but actually don't, the design flaws are obscured by ambiguity. Guido's earlier comment about "political correctness" underscores this point. This is not simply a case of me preferring my way of thinking about IP addresses to the way the ipaddr authors think about IP addresses. I'm simply stating a fact that, were it not true, the Internet would not function: addresses and networks are not the same thing. To represent them in the same class is a mistake.
But I think both of those can be implemented fairly trivially as subclasses of the existing ipaddr objects.
Yes, I could certainly see BaseIPAddress and BaseIPNetwork classes inheriting from BaseIP, with the ipaddr.IP function selectively return an object of either type depending on what the user passes in the constructor. If they include a mask, it's a network. If they don't, it's an address. I think that particular change might be backwards compatible, but I'm not sure. None the less, other changes that I think are important (returning IP objects instead of strings and ints for some properties; renaming poorly named methods) do change the API and thus are not backwards compatible without adding lots of cruft. Better to fix those now, in my opinion.
In short, netaddr's object model does not match my desired model, while ipaddr's is a lot closer to my desired model.
I'm not advocating netaddr. The decision of which library to use has already been made; I'm not debating that point. I'm merely suggesting that we pull ipaddr from the release until such time that it can be evolved to have a more agreeable API. ipaddr might be usable for you (probably because your limited understanding of IP matches the ipaddr developers'), but we've already heard from a handful of others on this list who would rather roll their own library than suffer through the quirks in ipaddr's current implementation. Clay
Clay McClure <clay <at> daemons.net> writes:
Not only useful, but necessary. It seems there are few people on this list who understand IP well enough to realize how distorted ipaddr actually is.
Not having myself enough knowledge about IP routing and administration (although I did happen to dissect BGP and IS-IS packets in an earlier life), I'm, however, inclined to trust Jean-Paul's word when it comes to network programming. IMHO, we should delay ipaddr's official inclusion in the stdlib until the conceptual issues are solved. Regards Antoine.
I've just subscribed to python-dev again after being pointed towards this thread (thanks Raymond). I'd be the first to accept failings and oddities in the interface of my own library. Since its release netaddr has taken its own interesting set of twists and turns. However, all along I have tried to be responsive and accommodating with regard to my users needs and requests for new features and bug fixes. There has been a lot of useful feedback which I have faithfully incorporated into netaddr's codebase. It would be a shame to see all the hard work go to waste on Y.A.P.I.M. :- http://code.google.com/p/netaddr/wiki/YetAnotherPythonIPModule There is a veritable graveyard of stuff out there! Some good, some not so good. The netaddr/ipaddr-py merge went awry and our projects decided to remain separate. I don't see any benefit in raking over those old coals; it's all water under the bridge. That being said, based on the passion in this thread, the decision to include ipaddr-py into the stdlib seems to be proving too hasty and contentious for some. It really might be worth taking a step back and taking heed of those concerns. Having had quite a while to think about this over the last few months, this is what I would ideally like to see. A sensible discussion from a *broad* base of network admins, sysadmins and developers using Python on the formulation of a reasonable functional specification and design for a brand new library (without the associated baggage and vested interests of any particular implementation). This would be an opportunity for us to nail our respective problems and differences while simultaneously bringing together most of the Python community behind code that WE WILL ACTUALLY USE. If this is going in the stdlib surely that is doubly important? If possible, I would particularly like to see input from Victor Stinner, the current IPy maintainer. There as some wrinkles and failings in IPy's interface and implementation but its actually not as bad as I first thought having actually spent some time implementing its interface (mostly successfully) as a facade on top of netaddr. See the netaddr repo if you are interested and *no* it is not supported code ;-) Would this actually be feasible or am I just a hopeless optimist? Should we formulate a PEP even though calls for that have so far been rejected? Possibly PEPs don't apply to libraries? Whoever overseas this would need to be someone with a fairly neutral perspective on all of this. Dave M.
I've just subscribed to python-dev again after being pointed towards this thread (thanks Raymond).
The same for me, thanks. :-) I'm the author of IPLib [1]; I don't consider myself an expert on the subject and my code dates back to Python 1.6 times (last updated in 2005). Moreover, while it works for me and its almost non-existent user base, I assume it can't be used for anything but as a source of ideas.
There is a veritable graveyard of stuff out there! Some good, some not so good.
The fact is that most of the times a programmer writes yet another Python IP module to cover only a very limited and usually simple aspect of IP manipulation that he needs at the given time (a conversion to/from decimal, a check for inclusion in a CIDR, ...)
A sensible discussion from a *broad* base of network admins, sysadmins and developers using Python on the formulation of a reasonable functional specification and design for a brand new library
To me, this makes a lot of sense: check which are the most used modules and ask the users. Generically speaking, I tend to agree with Clay, as I always had looked at IP addresses, netmasks and CIDR blocks as separate concepts, but again: I'm not an expert. +++ [1] http://erlug.linux.it/~da/soft/iplib/ Only supports IPv4; its main use is to convert amongst notations, but can be used to check if an IP (or another subnet) is included in a CIDR block and to gather some basic information about a CIDR. -- Davide Alberani <davide.alberani@gmail.com> [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/
On Mon, 1 Jun 2009 at 18:54, Jake McGuire wrote:
On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. L�wis" <martin@v.loewis.de>wrote:
As for Clay McLure's issue: I feel it's primarily a matter of taste. I see nothing morally wrong in using the same class for hosts and networks, i.e. representing a host as a network of size 1. I can understand why people dislike that, but I don't see why it would stop people from doing with the library what they want to do.
To the extent that Clay is having issues, it's because ipaddr.py is poorly documented, has potentially confusing comments, and he became confused. Lesser issues are that ipaddr.py doesn't work the way he wants and that ip addressing is inherently subtle.
Looking at the code in detail shows that ipaddr.IP/IPv4/IPv6 objects always represent *networks*. He wants one particular address out of that network, and that requires using __getitem__ or using IP.ip_ext. Neither is particularly intuitive.
import ipaddr ip = ipaddr.IPv4('10.33.11.17') ip IPv4('10.33.11.17/32') ip[0] '10.33.11.17'
Actually I find that very intuitive if I understand that the objects are always networks.
ip.ip_ext '10.33.11.17'
That's not intuitive :)
This feels much more like poor documentation than wide-ranging conceptual flaws.
I could put this in the tracker, but I'm not sure if that's appropriate.
I would say yes, put it in the tracker. --David
On Tue, Jun 2, 2009 at 9:22 AM, R. David Murray <rdmurray@bitdance.com> wrote:
ip[0]
'10.33.11.17'
Actually I find that very intuitive if I understand that the objects are always networks.
I suspect the authors would disagree with you on this point, since they insist that host routes and host addresses are the same thing, but assuming you are right, two flaws are immediately apparent: * The ipaddr classes are poorly named. Since they model networks, they should be named appropriately: IPNet or some such. * The ipaddr authors have overlooked IP addresses, which also deserve first-class treatment in any meaningful IP address library. It is called ipaddr, after all, and not ipnet.
ip.ip_ext
'10.33.11.17'
That's not intuitive :)
No, nor is the rest of the library intuitive. This is one of the reasons I've called it "quirky". Clay
participants (11)
-
"Martin v. Löwis" -
Antoine Pitrou -
Clay McClure -
Davide Alberani -
DrKJam -
Gregory P. Smith -
Guido van Rossum -
Jake McGuire -
Jake McGuire -
R. David Murray -
Raymond Hettinger