On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire@google.com> wrote:

On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido@python.org> wrote:

...

What's the opinion of the other interested party or parties? I don't want a repeat of the events last time, where we had to pull it at the last time because there hadn't been enough discussion.

How many other people are using this library?  I think it's hard to give really useful feedback on an API without using it for some non-trivial task, but maybe other people don't have this problem. -jake

188 (check that, 190) people have downloaded the 2.0 release in the last week (numbers publicly available from the code.google.com). I can't tell you how many (if any) have downloaded it via svn. Cheers, /peter

On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire <mcguire@google.com> wrote:

On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody <peter@hda3.com> wrote:

...

On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire@google.com> wrote:

...

On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido@python.org> wrote:

...

What's the opinion of the other interested party or parties? I don't want a repeat of the events last time, where we had to pull it at the last time because there hadn't been enough discussion.

How many other people are using this library?  I think it's hard to give really useful feedback on an API without using it for some non-trivial task, but maybe other people don't have this problem. -jake

188 (check that, 190) people have downloaded the 2.0 release in the last week (numbers publicly available from the code.google.com). I can't tell you how many (if any) have downloaded it via svn.

Downloading and using are not the same thing.

Correct, but there is a strong positive correlation between the two. If you have a better method for determining what you would consider an appropriate level of usage, I'm all ears.

-jake

On Tue, Sep 15, 2009 at 11:44 AM, Jake McGuire <mcguire@google.com> wrote:

On Tue, Sep 15, 2009 at 11:36 AM, Peter Moody <peter@hda3.com> wrote:

...

...

On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody <peter@hda3.com> wrote:

...

On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire@google.com>

wrote:

...

...

On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido@python.org> wrote:

...

What's the opinion of the other interested party or parties? I don't want a repeat of the events last time, where we had to pull it at

On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire <mcguire@google.com> wrote: the

...

...

...

...

last time because there hadn't been enough discussion.

How many other people are using this library? I think it's hard to give really useful feedback on an API without using it for some non-trivial task, but maybe other people don't have this problem. -jake

188 (check that, 190) people have downloaded the 2.0 release in the last week (numbers publicly available from the code.google.com). I can't tell you how many (if any) have downloaded it via svn.

Downloading and using are not the same thing.

Correct, but there is a strong positive correlation between the two. If you have a better method for determining what you would consider an appropriate level of usage, I'm all ears.

Put something on the project page (or download page if possible) saying "ipaddr is being considered for inclusion in the Python standard library. We want to make sure it meets your needs, but we need you to tell us. If you use ipaddr and like it, please let us know on ip-addr-dev."

I dunno, maybe it's too much work. But no one else seems to have an opinion strong enough to share, at least not at this point.

To clarify: there will always be some group of people happy to tell you that whatever you made is crap and that it should have been done differently. There may be many more people out there who think that what you did is great. But those people probably don't read python-dev so their voices don't get heard when it comes to deciding what to do with PEP 3144. The easiest way to get their voices heard is to ask them to speak up, and the one place you know they visited was the ipaddr page on code.google.com, so you may as well ask them there. -jake

...

...

188 (check that, 190) people have downloaded the 2.0 release in the last week (numbers publicly available from the code.google.com). I can't tell you how many (if any) have downloaded it via svn.

Downloading and using are not the same thing.

Correct, but there is a strong positive correlation between the two. If you have a better method for determining what you would consider an appropriate level of usage, I'm all ears.

A good way of determining the level of usage would be pointing to open source projects that are popular in the python community and which incorporate your module. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown

...

...

...

...

188 (check that, 190) people have downloaded the 2.0 release in the last week (numbers publicly available from the code.google.com). I can't tell you how many (if any) have downloaded it via svn.

Downloading and using are not the same thing.

Correct, but there is a strong positive correlation between the two. If you have a better method for determining what you would consider an appropriate level of usage, I'm all ears.

A good way of determining the level of usage would be pointing to open source projects that are popular in the python community and which incorporate your module.

well, the 2.0 release is still new. codesearch.google.com shows some projects using the 1.x release; hopefully some of those 200 downloaders will put up some publicly indexable python code at some point.

I think one first needs to wait until this happens, I meana large user base is formed, before a meaningful discussion can be done on whether to include it in the stdlib or not. The long and largely academic thread here I think illustrates this point. Without a large user base it's up to anybody's gut feelings what is 'right' and what 'feels wrong'. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown

On Thu, Sep 17, 2009 at 11:10 AM, Daniel Fetchinson <fetchinson@googlemail.com> wrote:

...

...

...

...

...

188 (check that, 190) people have downloaded the 2.0 release in the last week (numbers publicly available from the code.google.com). I can't tell you how many (if any) have downloaded it via svn.

Downloading and using are not the same thing.

Correct, but there is a strong positive correlation between the two. If you have a better method for determining what you would consider an appropriate level of usage, I'm all ears.

A good way of determining the level of usage would be pointing to open source projects that are popular in the python community and which incorporate your module.

well, the 2.0 release is still new. codesearch.google.com shows some projects using the 1.x release; hopefully some of those 200 downloaders will put up some publicly indexable python code at some point.

I think one first needs to wait until this happens, I meana large user base is formed, before a meaningful discussion can be done on whether to include it in the stdlib or not. The long and largely academic thread here I think illustrates this point. Without a large user base it's up to anybody's gut feelings what is 'right' and what 'feels wrong'.

I like the sound of this. I'm going to continue to work on the PEP and the reference implementation, but as Martin said earlier, there's no reason to hurry. any discussion on ipaddr and/or PEP 3144 can continue on ipaddr-py-dev@googlegroups.com in the meantime (past spam unfortunately means that you have to subscribe in order to post, apologies, but all are welcome). Hopefully, the continued work will help iron any remaining confusion of the API. Cheers, /peter

Cheers, Daniel

-- Psss, psss, put it down! - http://www.cafepress.com/putitdown _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

Andrew McNamara <andrewm <at> object-craft.com.au> writes:

>>> ipaddr.IPv4Network('192.168.1.1/16').network IPv4Address('192.168.0.0')

Er, does this mean that taking the `network` attribute from a network object actually gives an address object (not a network)? It looks horribly misleading to me.

R. David Murray wrote:

On Tue, 15 Sep 2009 at 14:28, Antoine Pitrou wrote:

...

Andrew McNamara <andrewm <at> object-craft.com.au> writes:

...

...

...

...

ipaddr.IPv4Network('192.168.1.1/16').network IPv4Address('192.168.0.0')

Er, does this mean that taking the `network` attribute from a network object actually gives an address object (not a network)? It looks horribly misleading to me.

That was my opinion, too. I've always called that number the 'zero' of the network, but other people said that it is usually called the 'network'.

I also find it odd to see a 'network' with an IP address of 192.168.1.1. That looks like a host-address-plus-netmask to me, not a network address.

It just happened that I needed a tool today to calculate the gateway IP for an interface, and I took it as an opportunity to try out this ipaddr module. I'll attempt to relate my experience below... I have to concur with the opinions above. I was very confused by the following error:

...

...

addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0") ... ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not a valid address (hint, it's probably a network)

Because, it *is* a address of a host on a network. I gave in and said:

...

...

net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")

But then, I was dumbfounded as to how I could get the gateway IP from this IPNetwork object. It took me a while to figure out that you can iterate over IPNetwork instances:

...

...

gateway = net[1]

I was then confused, because:

...

...

print(type(gateway)) <class 'ipaddr.IPv4Address'>

Which sorta blew my mind.. I fully expected to receive an IPNetwork back from that operation. It is unclear to me why the network information gets chucked by that operation. I foresee having to work around that in real applications by doing something obnoxious like:

...

...

actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask))

But since I only actually needed the IP address for the gateway, it fulfilled my needs. In the end, I found the names IPNetwork/IPAddress and their instantiations confusing. ISTM that IPNetwork is overloaded and plays two roles of being an IPNetwork and being an IPAddressWithNetwork. And finally, it's unclear to me why iterating over a IPNetwork would not produce a sequence of IPNetwork(/IPAddressWithNetwork) objects. ISTM that if I started with an IPNetwork object, the API should always return IPNetwork objects. If I want just an IPAddress object, then I can always fetch the "ip" attribute to get that. Perhaps there is some compelling conceptual argument as to why this is not correct, but it seems like the API destroys information needlessly. Just my opinion.. -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

Peter Moody wrote:

On Tue, Sep 15, 2009 at 10:16 AM, Scott Dial <scott+python-dev@scottdial.com> wrote:

...

In the end, I found the names IPNetwork/IPAddress and their instantiations confusing. ISTM that IPNetwork is overloaded and plays two roles of being an IPNetwork and being an IPAddressWithNetwork. And finally, it's unclear to me why iterating over a IPNetwork would not produce a sequence of IPNetwork(/IPAddressWithNetwork) objects.

What you're describing, at least for the networks, is basically what ipaddr was. It seemed, after much heated discussion, that the community felt that

ipaddr.IPv4Network('1.1.1.0/24')[0]

was actually an individual address. ie, each of the addresses in 1.1.1.0/24 are individual addresses, rather than networks with /32 prefix lengths.

For clarity, I am going to call the current design "A": ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Address('1.1.1.0') And what itt sounds like what you are describing as the old behavior is this (design "B"): ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/32') Which is different than what I was trying to describe. I expected (design "C"): ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/24') My summarization of these designs would be that "B" is wrong. And that "A" is better than "B", certainly. At least "A" is not saying something untrue. However, "C" would be truthful and retains a superset of information that "A" provides (the "ip" attribute of IPNetwork). In other words, I don't see why obtaining a host address would *not* retain the hostmask from the network it was obtained from. I am not disagreeing with it being an individual address. I am disagreeing that IPNetwork itself already does represent individual addresses (hence my aliasing it with IPAddressWithNetwork). And wrt, the logical return would be another IPAddressWithNetwork retaining the same mask. -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

R. David Murray <rdmurray <at> bitdance.com> writes:

x = IPv4AddressWithMask('192.168.1.1/24') x.network == IPv4Network('192.168.1.0/24') x.network[1] == x

I don't think we need an IPAddressWithMask which would just complicate the API without any obvious benefit. We just need a factory function which returns a tuple after parsing: >>> addr, net = parse_address_with_mask('192.168.1.1/24') >>> addr == IPv4Address('192.168.1.1') True >>> net == IPv4Network('192.168.1.0/24') True Regards Antoine.

R. David Murray <rdmurray <at> bitdance.com> writes:

I would find that acceptable but sub-optimal. Most of my use cases (which involve manipulating router and firewall configuration files) would then start by making a little class named AddressWithNetwork to hold the tuple returned by your parse function, with attributes 'ip' and 'network' and a representation that included the netmask.

If the AddressWithNetwork class has useful functionality of its own, then why not (although I think that conceptually, this functionality should belong to the Network class instead). If, however, the AddressWithNetwork class is only a recipient for a tuple, then it sounds useless. It's like having an AddressWithPort to hold things like "127.0.0.1:8080". Building little featureless recipients like that isn't really idiomatic Python, IMO.

An IPv4Address with 'network' and 'mask' attributes that could be None would also not complicate the API, IMO, and would handle both of these use cases.

But it would be confusing and conceptually bizarre, because an address (in usually accepted terminology) doesn't have a network and a mask. I think we should keep the API clean and reasonably logical, rather than try to cover all use cases at the expense of weird shortcuts. Regards Antoine.

Le mardi 15 septembre 2009 à 15:48 -0400, R. David Murray a écrit :

It's useful functionality is parsing/validating an address+mask, rendering as address+mask, and being able to get the associated IP and network objects from it. It's a small class, but useful, IMO.

If it's only about parsing and validating, then a tuple works just fine. Getting the associated IP objects is obtained from iterating over the network. Getting the associated network objects I don't understand. There is a single network, not a bunch of them.

However, I do not think that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') as valid. That's just too confusing and error prone.

Indeed, it should throw some kind of ValueError instead.

Oh, and if people don't like 'zero' as the name for the IPv4Address at the beginning of the network IP range, how about 'network_ip' or 'network_address' instead of just network, to make it clearer that it is an address?

How about something more explicit about how it's obtained, like 'lower_bound'?

R. David Murray wrote:

On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote:

...

Le mardi 15 septembre 2009 à 15:48 -0400, R. David Murray a écrit :

...

However, I do not think that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') as valid. That's just too confusing and error prone.

Indeed, it should throw some kind of ValueError instead.

Peter, what do you think?

This change would have to be dependent on Antoine's suggestion of: >>> addr, net = parse_address_with_mask('192.168.1.1/24') >>> addr == IPv4Address('192.168.1.1') True >>> net == IPv4Network('192.168.1.0/24') True Otherwise, I am ok with this change too. It resloves the weird duality of IPv4Network. And as RDM says, anyone who wants the IPv4AddressWithNetwork functionality can just role it themselves. At the moment of iteration, you have access to what network it is and can build your own IPv4AddressWithNetwork objects. net = IPv4Network('10.0.0.0/24') netaddrs = [] for addr in net: netaddrs.append(IPv4AddressWithNetwork(addr, net)) I guess I am ok with this. It seems sub-optimal (why not just return a IPv4AddressWithNetwork to begin with?) but I suppose it is no less efficient since the same objects would be constructed. -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

Peter Moody <peter <at> hda3.com> writes:

...

...

...

...

However, I do not think that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') as valid.  That's just too confusing and error prone.

Indeed, it should throw some kind of ValueError instead.

Peter, what do you think?

I disagree. It seems overly pedantic to throw a valueerror on this. IPy does (used to do?) this and it's one of the primary reasons I wrote ipaddr.

Python is not PHP and does not try to be overly "smart" or tolerant when faced with bizarrely-formatted input. I don't see any valid reason for entering a network as "192.168.1.1/24" rather than the canonical "192.168.1.0/24". The former might indicate a typing error or a mental slip, so let's be helpful and signal it to the user.

for addr in ipaddr.IPv4Network('1.1.1.0/24').iterwithnetmask(): # 1.1.1.0/24, 1.1.1.1/24, 1.1.1.2/24...

I don't have any strong feelings about it, although I don't really think it is needed, and a plethoric API is not a good thing (neither clarity-wise nor maintenance-wise).

this can be done much more cleanly/easily than adding two new (redundant in my eyes) classes and much less pedantically than raising exceptions w/ IPv?Network() constructor received something other than base network address.

I don't think anybody suggested adding two classes. As for the "pedantry" of thoroughly validating inputs, see above. Regards Antoine.

Antoine Pitrou wrote:

Antoine Pitrou <solipsis <at> pitrou.net> writes:

...

I don't see any valid reason for entering a network as "192.168.1.1/24" rather than the canonical "192.168.1.0/24". The former might indicate a typing error or a mental slip, so let's be helpful and signal it to the user.

Or perhaps there can be an optional "strict=True" (or "strict=False") argument to the constructor / parsing function.

If so, I hope that we'd default to strict=True.

On Tue, Sep 15, 2009 at 5:33 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Antoine Pitrou <solipsis <at> pitrou.net> writes:

...

I don't see any valid reason for entering a network as "192.168.1.1/24" rather than the canonical "192.168.1.0/24". The former might indicate a typing error or a mental slip, so let's be helpful and signal it to the user.

Or perhaps there can be an optional "strict=True" (or "strict=False") argument to the constructor / parsing function.

I can live w/ a default of strict=False. there are plenty of cases where it's not an error and easy enough ways to check, if the developer is concerned, with or without an option. eg if addr.ip != addr.network: Cheers, /peter

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

I'd like to see more clear direction for this. Is it a string manipulation library, or an IP address library? There are ONLY TWO types of IP representations: 1. Network = 32 bit number AND 32 bit mask. That is the binary AND. The mask needs to be preserved, but not the network portion. Any library should return the ANDed result. Whether the number is 192.186.1.1/24 or 192.168.1.0/24, the result after 'and' is the same. Add a strict=false in case people need it for validation, better yet, add a validation helper function, but always return the proper form. 2. A host address is an IP address with no network info. No mask information, ever. What about Broadcast / Network IDs? The "Network-ID or Network Number" and "Broadcast" are meaningless without context. 1. A network number by itself it's just another address. Its use depends on context. When routing, it needs a mask.

route 10.0.0.0 255.0.0.0 192.168.1.1

When writing firewall ACLs, it is a host.

access-list aclname permit ip any 192.168.1.2 255.255.255.254

Since both are valid, preference shouldn't be given. Instead add a helper for firstip. 2. Same story with broadcast. Use depends on context. Add the lastip helper. What does iterating over a network mean? Iterating over the set of addresses in a network must return host addresses. If it returns a network, then the address should be ANDed, but that would return the same number for every iteration. Only returning hosts makes sense. We need to set a solid line between IP String manipulation and IP manipulation. If you want an IP with a mask that isn't a network, then you're not doing IP address manipulation, you're doing string manipulation. There are many use cases for it, but it's a separate consideration to pure IP manipulation. If string manipulation is desirable, then add another standard library called ipstring (or a separate class if you want it all jumbled in the same import). This way real network computing can be done without weird quirks of representation. R.

On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith <eric@trueblade.com> wrote:

Antoine Pitrou wrote:

...

Peter Moody <peter <at> hda3.com> writes:

...

...

...

...

> > However, I do not think > that the proposed API should accept, eg, > IPv4Network('192.168.1.1/24') > as valid.  That's just too confusing and error prone.

Indeed, it should throw some kind of ValueError instead.

Peter, what do you think?

I disagree. It seems overly pedantic to throw a valueerror on this. IPy does (used to do?) this and it's one of the primary reasons I wrote ipaddr.

Python is not PHP and does not try to be overly "smart" or tolerant when faced with bizarrely-formatted input.

I don't see any valid reason for entering a network as "192.168.1.1/24" rather than the canonical "192.168.1.0/24". The former might indicate a typing error or a mental slip, so let's be helpful and signal it to the user.

I completely agree. I don't know of any situation where I'd want a network of "192.168.1.1/24" to be anything other than an error.

when you're entering the address of your nic.

Eric. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

2009/9/16 Steven D'Aprano <steve@pearwood.info>:

I've been skimming emails in this thread, since most of them go over my head and I have no current need for an ipaddress module.

Same here.

As an outsider to this argument, and judging from the lack of agreement here, it seems to me that some (many? most?) developers who have need of ipaddress functions are a little unclear about the difference, or lack thereof, between addresses and networks. Is it our aim to pander to such confusion, and have a module which will Just Work for such users, even at the risk of sometimes accepting invalid input. Or is it to have a module which is strict and forces the user to Do The Right Thing, even at the cost of being less easy to use?

For what it's worth, it seems to me that it would be better to have a strict module in the standard lib, and leave the DWIM code to third parties.

As a non-expert, I find the confusion between network, address, address with mask, etc to be extremely unhelpful. I haven't looked to confirm if it's a confusion that only exists in the discussion, or if it's in the code as well. However, from my POV, I'd rather have a module that made the concepts clear and unambiguous, effectively educating me in the details of correct usage while I used it, rather than one that catered to my muddled thinking and allowed me to remain confused. Of course, the discussion seems to imply that even the experts have a confused view, so maybe I'm being too ambitious here :-) Paul.

Steven D'Aprano wrote:

I've been skimming emails in this thread, since most of them go over my head and I have no current need for an ipaddress module. But one thing I noticed stands out and needs commenting on:

On Wed, 16 Sep 2009 11:05:26 am Peter Moody wrote:

...

On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith <eric@trueblade.com> wrote:

...

I completely agree. I don't know of any situation where I'd want a network of "192.168.1.1/24" to be anything other than an error. when you're entering the address of your nic.

Eric is talking about a network. Peter replies by talking about an address.

Martin explained it better in another part of the thread:

if you know your address is 82.94.164.162, how do you compute what you should spell for 82.94.164.162/27?

Or, to put it another way, given an arbitrary host in a network (e.g. your own machine or the default gateway) and the netmask for that network, calculate the network address. With a "lax" parser on IPNetwork this is a trivial task - just create the network object and then retrieve the network address from it. If, on the other hand, IPNetwork demands that you already know the network address before allowing you to create an IPNetwork object, then you're pretty much out of luck - if all you have to work with are the IP strings then this is actually a tricky calculation. If the default IPNetwork constructor was made more strict, then this functionality would have to be made available another way (probably as an alternate constructor like IPNetwork.from_host_address rather than as a boolean 'strict' option) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

exarkun@twistedmatrix.com wrote:

On 11:10 am, ncoghlan@gmail.com wrote:

...

If the default IPNetwork constructor was made more strict, then this functionality would have to be made available another way (probably as an alternate constructor like IPNetwork.from_host_address rather than as a boolean 'strict' option)

This seems to be the right solution to me, particularly the use of an alternate constructor rather than an ambiguously named flag.

Particular since someone pointed out that the "Network" object was remembering which IP address happened to be passed to the constructor (which is "Address with Mask" behaviour, as has been mentioned). That said, I've been trying to refrain from commenting on specifics of the API, as I haven't tried to use it for anything. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Nick Coghlan wrote:

Or, to put it another way, given an arbitrary host in a network (e.g. your own machine or the default gateway) and the netmask for that network, calculate the network address.

Some people have claimed that the gateway address of a network isn't necessarily the zero address in that network. If that's true, then you *can't* calculate the network address from a host address and a netmask -- there isn't enough information. Furthermore, an IPNetwork object needs to be able to represent a network address whose address part contains bits that aren't in the mask. -- Greg

R. David Murray wrote:

A network is conventionally represented by an IP address in which the bits corresponding to the one bits in the netmask are set to zero, plus the netmask.

Okay, that's clarified things for me, thanks. In that case, we shouldn't be talking about a "network address" at all, but just a "network", and it makes sense to have 1) A class called IPNetwork that contains an IP number and a mask, with the IP number constrained not to have any ones where the mask has zeroes, and 2) A class called IPAddress which only contains an IP number. It seems that some people would also like to have 3) A class called IPAddressWithMask that contains an IP number and a mask, with no constraints, but I'm not enough of an expert to know whether such a thing would be used often enough to be worth having a special type for it, as opposed to using an (IPNetwork, IPAddress) pair, or attaching a 'network' attribute to an IPAddress, or some other solution when the need arises. -- Greg

On Wed, Sep 16, 2009 at 5:30 PM, Andrew McNamara <andrewm@object-craft.com.au> wrote:

...

R. David Murray wrote:

...

A network is conventionally represented by an IP address in which the bits corresponding to the one bits in the netmask are set to zero, plus the netmask.

Okay, that's clarified things for me, thanks.

Put another way, an "Address" describes a single end-point and a "Network" describes a set of (contiguous) Addresses.

Where things have become confused is that, for practical reasons, it is convenient to have a representation for an Address and it's containing Network (the later can be derived from the Address and a mask). We tried to make the current Network entity do double-duty, but it is just leading to confusion.

This is why I proprose there be three entities:

 * an Address entity (same as the current one)  * a Network entity (like now, but requires masked bits to be zero)  * an AddressWithMask entity (existing Network, but no container behaviour)

This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6). It's still unclear to me what is gained by pulling AddressWithMask functionality out of the current network classes. It's easy enough for the concerned developer who to check if the entered network address does actually have all of its host bits set to zero. It is not my experience that this behavior is desired so often that having the network classes behave as they do now leads to a great deal of confusion.

There is a school of thought that says we only need a single class that behaves like the current Network entity - end-points are simply represented by an all-ones mask. This is, I think, where we started. But this scheme was rejected.

I don't think anything has actually been rejected.

-- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Wed, Sep 16, 2009 at 6:32 PM, Andrew McNamara <andrewm@object-craft.com.au> wrote:

...

This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6).

Yes, I know - I was just trying to keep to the point.

...

It's still unclear to me what is gained by pulling AddressWithMask functionality out of the current network classes. It's easy enough for the concerned developer who to check if the entered network address does actually have all of its host bits set to zero. It is not my experience that this behavior is desired so often that having the network classes behave as they do now leads to a great deal of confusion.

I think we're in a painful middle ground now - we should either go back to the idea of a single class (per protocol), or make the distinctions clear (networks are containers and addresses are singletons).

Personally, I think I would be happy with a single class (but I suspect that's just my laziness speaking). However, I think the structure and discipline of three classes (per protocol) may actually make the concepts easier to understand for non-experts.

I think this is where we disagree. I don't think the added complexity does make it any easier to understand.

A particular case in point - if you want to represent a single IP address with netmask (say an interface), you use a Network class, not an Address class. And the .network attribute returns a Address class!

Right, and I don't see where the confusion lies. You have an address + netmask. ergo, you have a Network object. The single address that defines the base address (most commonly referred to as the network address) is an Address object. there is no netmask associated with that single address, ergo, it's an Address object.

The reason I suggest having the Network class assert that masked bits be zero is two-fold:

 * it ensures the correct class is being used for the job  * it ensures application-user errors are detected as early as possible

I also suggest the AddressWithMask classes not have any network/container behaviours for a similar reason. If the developer needs these, the .network attribute is only a lookup away.

the problem I have with this approach is that it seems like a long way to go for a shortcut (of checking if addr.ip != addr.network: raise Error). Cheers, /peter

-- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Wed, Sep 16, 2009 at 7:48 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Peter Moody wrote:

...

I don't see where the confusion lies.  You have an address + netmask. ergo, you have a Network object.  The single address that defines the base address (most commonly referred to as the network address) is an Address object. there is no netmask associated with that single address, ergo, it's an Address object.

But if I understand correctly, you *don't* have a network, you have something representing a particular host address within a network, plus information that lets you deduce the network to which it belongs. So calling the object a 'Network' is a misnomer.

What's more, I don't see the point of having a 'network' attribute without a mask, because the zero address of a network on its own doesn't define the network. You need the zero address plus a mask to do that.

I'm not sure what usefulness the zero address on its own has, but if it's considered useful enough to have an attribute for it, calling it something like 'base_address' would be less confusing.

the address with all of the hosts bits masked to zero is most commonly referred to as the network address. same as the address with all of the host bits set to one is called the broadcast address. calling it something like base_address or min_address will cause quite a bit more confusion.

-- Greg _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

Antoine Pitrou wrote:

Peter Moody <peter <at> hda3.com> writes:

...

the address with all of the hosts bits masked to zero is most commonly referred to as the network address. same as the address with all of the host bits set to one is called the broadcast address. calling it something like base_address or min_address will cause quite a bit more confusion.

Quite a bit less IMO. I'm not a network specialist, but the "network address"

Nah, network address is perfectly explicit - it's what you get when you bitwise and any host within that network with the netmask. Where it becomes confusing is that we have a property called "network" that returns an IPAddress object rather than an IPNetwork object. People that are network specialists would grasp that fairly easily, but we try to design standard library APIs so that they're novice friendly as well. And the above situation isn't novice friendly. To be honest, given the indexing behaviour, I'm -1 on the idea of giving the network address or broadcast address attribute names *at all*. Consider: network_address = my_net[0] broadcast_address = my_net[-1] The only way the network address could match the number of characters in the indexing method is if you just called it the network id (i.e. my_net.id). For the broadcast address, there is no name that even comes close to matching the indexing approach for brevity. And since IPNetwork is a container for IPAddress instances, the indexing approach means there is zero ambiguity regarding the return type. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Thu, Sep 17, 2009 at 6:25 AM, Eric Smith <eric@trueblade.com> wrote:

Nick Coghlan wrote:

...

To be honest, given the indexing behaviour, I'm -1 on the idea of giving the network address or broadcast address attribute names *at all*. Consider:

 network_address = my_net[0]  broadcast_address = my_net[-1]

My only concern with this is a possible performance issue with v6 networks. Would this be implemented such that [-1] doesn't need to iterate through the (possibly large) address space of a v6 network?

I think it could (and would) be implemented efficiently, so I'm +1 on not giving these names, but adding this "recipe" to the documentation instead.

indexing is plenty efficient, but the problem is that these names for these attributes are common to the point of causing confusion if they're omitted. Cheers, /peter

Eric.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

Eric Smith wrote:

Peter Moody wrote:

...

indexing is plenty efficient, but the problem is that these names for these attributes are common to the point of causing confusion if they're omitted.

After thinking about it some more, I'm okay with names for [-1] and [0]. I like .broadcast, and I can live with .network (although it's confusing, since in my view it should return an address, not a network).

And in fact *does* return an Address - the same address that my_net[0] returns. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Cameron Simpson wrote:

On 18Sep2009 07:48, Nick Coghlan <ncoghlan@gmail.com> wrote: | Eric Smith wrote: | > Peter Moody wrote: | >> indexing is plenty efficient, but the problem is that these names for | >> these attributes are common to the point of causing confusion if | >> they're omitted. | > | > After thinking about it some more, I'm okay with names for [-1] and [0]. | > I like .broadcast, and I can live with .network (although it's | > confusing, since in my view it should return an address, not a network). | | And in fact *does* return an Address - the same address that my_net[0] | returns.

Yes, I think Eric was complaining about the name being "network", since we have Network objects and this isn't one.

Right. thing_commonly_referred_to_as_network_but_is_an_address.

On Thu, 17 Sep 2009 10:40:47 pm Nick Coghlan wrote:

Antoine Pitrou wrote:

...

Peter Moody <peter <at> hda3.com> writes:

...

the address with all of the hosts bits masked to zero is most commonly referred to as the network address. same as the address with all of the host bits set to one is called the broadcast address. calling it something like base_address or min_address will cause quite a bit more confusion.

Quite a bit less IMO. I'm not a network specialist, but the "network address"

Nah, network address is perfectly explicit - it's what you get when you bitwise and any host within that network with the netmask.

[soapbox] I really wish people would stop misusing the word "explicit" to mean (apparently) "I know how this is commonly defined". To a non-specialist, "the network address" is ambiguous. There are many addresses in a network, and none of them are the entire network. It's like saying, given a list [2, 4, 8, 12], what's "the list item"? If network specialists have a convention of calling hostadd&netmask "the network address", that's fine. It's jargon which seems illogical to me, but if it's useful and in common use, specialists will expect us to support it. But it's not explicit, it's a definition.

Where it becomes confusing is that we have a property called "network" that returns an IPAddress object rather than an IPNetwork object.

People that are network specialists would grasp that fairly easily, but we try to design standard library APIs so that they're novice friendly as well. And the above situation isn't novice friendly.

+1000 on this!

To be honest, given the indexing behaviour, I'm -1 on the idea of giving the network address or broadcast address attribute names *at all*. Consider:

network_address = my_net[0] broadcast_address = my_net[-1]

The only way the network address could match the number of characters in the indexing method is if you just called it the network id (i.e. my_net.id). For the broadcast address, there is no name that even comes close to matching the indexing approach for brevity.

Is brevity so much more important than clarity and discoverability? -- Steven D'Aprano

On Fri, 18 Sep 2009 at 11:04, Andrew McNamara wrote:

[attribution lost; apparently Steven D'Aprano given the CC]

...

To a non-specialist, "the network address" is ambiguous. There are many addresses in a network, and none of them are the entire network. It's like saying, given a list [2, 4, 8, 12], what's "the list item"?

A "network address" is an IP address and mask, but I understand your confusion - we're mixing terminology from disperate domains. In my postings, I have tried to refer to Network (a containter) and Address (an item).

Apparently not, in many people's vocabulary. The 'network address' is used to designate the IP address whose bits corresponding to the one bits in the mask have been set to zero (ie: the first IP address in the network range). It is interesting how this item seems to lead to the greatest amount of confusion on everyone's part, and I'm guessing it is because the common terminology and usage blurs the line between addresses and networks. And that's what we are trying to make clear(er) through a well structured API. --David

On Fri, 18 Sep 2009 11:04:46 am Andrew McNamara wrote:

...

To a non-specialist, "the network address" is ambiguous. There are many addresses in a network, and none of them are the entire network. It's like saying, given a list [2, 4, 8, 12], what's "the list item"?

A "network address" is an IP address and mask, but I understand your confusion - we're mixing terminology from disperate domains. In my postings, I have tried to refer to Network (a containter) and Address (an item).

So to clarify, how many different things which need to be handled are there? Items: 1 IP address -- a 32 bit (IPv4) or 128 bit (IPv6) number 2 Netmask -- a bit mask of the form 111..100..0 3 Network address -- the lowest address in a network, and equal to (defined by?) the bitwise-AND of any address in the network with the network's netmask 4 Host address -- the part of the IP address that is not masked by the netmask 5 Broadcast address -- the highest address in a IPv4 network Containers: 6 Network -- a range of IP address Have I missed anything or got anything wrong? -- Steven D'Aprano

On Wed, 16 Sep 2009 at 20:26, Peter Moody wrote:

On Wed, Sep 16, 2009 at 7:48 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

...

I'm not sure what usefulness the zero address on its own has, but if it's considered useful enough to have an attribute for it, calling it something like 'base_address' would be less confusing.

the address with all of the hosts bits masked to zero is most commonly referred to as the network address. same as the address with all of the host bits set to one is called the broadcast address. calling it something like base_address or min_address will cause quite a bit more confusion.

If 'network address' is the most common designation for this number (and it may well be), then 'network.address' and 'network.broadcast' would seem to be the best choices for names. But there is definitely an attraction to the idea (I think it was Nick's?) of dropping these attributes entirely and just using indexing to get the addresses. That will make perfect sense to network professionals, and by putting appropriate words in the documentation will educate people new to IP addressing concepts. The advantage is that it is completely unambiguous: the network address really is the first address in the network range, and the broadcast really is the last, in common usage. Exceptional cases can then be handled by custom subclasses, without having someone who has to handle weird broadcast addresses (for example) submitting a bug request to make '.broadcast' settable. --David

On Thu, Sep 17, 2009 at 3:27 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Peter Moody wrote:

...

the address with all of the hosts bits masked to zero is most commonly referred to as the network address.

Then call the attribute 'network_address', not just 'network'.

from an earlier email: Again, the same error-catching functionality can be obtained through an option to the constructor. network and broadcast attributes can be renamed to .\1_address to alleviate confusion as well.

-- Greg _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara <andrewm@object-craft.com.au> wrote:

...

...

I think we're in a painful middle ground now - we should either go back to the idea of a single class (per protocol), or make the distinctions clear (networks are containers and addresses are singletons).

Personally, I think I would be happy with a single class (but I suspect that's just my laziness speaking). However, I think the structure and discipline of three classes (per protocol) may actually make the concepts easier to understand for non-experts.

I think this is where we disagree. I don't think the added complexity does make it any easier to understand.

I argue that we're not actually adding any complexity: yes, we add a class (per protocol), but we then merely relocate functionality to clarify the intended use of the classes.

And I argue the moving this functionality to new classes (and adding new restrictions to existing classes) doesn't buy anything in the way of overall functionality of the module or a developer's ability to comprehend intended uses.

...

...

A particular case in point - if you want to represent a single IP address with netmask (say an interface), you use a Network class, not an Address class. And the .network attribute returns a Address class!

Right, and I don't see where the confusion lies.

I suggest you are too close to the implementation to be surprised by it. 8-)

touche :)

...

You have an address + netmask. ergo, you have a Network object.

In a common use case, however, this instance will not represent a network at all, but an address. It will have container-like behaviour, but it should not (this is a property of networks, not addresses). So the instance will be misnamed and have behaviours that are, at best, misleading.

...

The single address that defines the base address (most commonly referred to as the network address) is an Address object. there is no netmask associated with that single address, ergo, it's an Address object.

I would argue that a Network never has a single address - by definition, it has two or more nodes. A .network attribute should return a Network instance. If you want the base address, this probably should be called .base_address or just .address (to parallel the .netmask attribute).

.network is shorthand for network address. are .network_address and .broadcast_address less confusing? I have to say, though, .network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4 use, IIRC .network...)

...

...

The reason I suggest having the Network class assert that masked bits be zero is two-fold:

* it ensures the correct class is being used for the job * it ensures application-user errors are detected as early as possible

I also suggest the AddressWithMask classes not have any network/container behaviours for a similar reason. If the developer needs these, the .network attribute is only a lookup away.

the problem I have with this approach is that it seems like a long way to go for a shortcut (of checking if addr.ip != addr.network: raise Error).

This isn't about shortcuts, but about correctness... having the Network object represent a network, and having Address objects represent end-points, and having errors discovered as early as possible.

Then what I don't see is the purpose of your network-only-network-object. essentially identical functionality can be obtained with the module as is w/o the added complexity of new classes. Cheers, /peter

What I'm arguing here is that singletons should not simultaneously be containers - it's not pythonic, and it leads to ambiguity. The underlying IP concepts don't require it either: an IP address is a singleton, a network is a container, and there is no overlap. Yes, an address may be a member of a network, and having a reference to that network on the address object is valuable, but the address should not behave like a network.

-- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

Peter Moody wrote:

On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara <andrewm@object-craft.com.au> wrote:

...

...

...

I think we're in a painful middle ground now - we should either go back to the idea of a single class (per protocol), or make the distinctions clear (networks are containers and addresses are singletons).

Personally, I think I would be happy with a single class (but I suspect that's just my laziness speaking). However, I think the structure and discipline of three classes (per protocol) may actually make the concepts easier to understand for non-experts. I think this is where we disagree. I don't think the added complexity does make it any easier to understand. I argue that we're not actually adding any complexity: yes, we add a class (per protocol), but we then merely relocate functionality to clarify the intended use of the classes.

And I argue the moving this functionality to new classes (and adding new restrictions to existing classes) doesn't buy anything in the way of overall functionality of the module or a developer's ability to comprehend intended uses.

Speaking as the originator of this thread of discourse, the lack of a 3rd class was exactly the source of my confusion and my inability to communicate my confusion to everyone. I clearly did not understand the intended uses of the IPNetwork type because it was capable of playing two roles that are decidedly different conceptually. So, I must respectfully disagree with you.

...

...

You have an address + netmask. ergo, you have a Network object. In a common use case, however, this instance will not represent a network at all, but an address. It will have container-like behaviour, but it should not (this is a property of networks, not addresses). So the instance will be misnamed and have behaviours that are, at best, misleading.

This is exactly the confusion and duality that I struggled with.

...

This isn't about shortcuts, but about correctness... having the Network object represent a network, and having Address objects represent end-points, and having errors discovered as early as possible.

Then what I don't see is the purpose of your network-only-network-object. essentially identical functionality can be obtained with the module as is w/o the added complexity of new classes.

The purpose is to avoid conflating IPNetwork with an IPAddress that has a mask. If the IPNetwork didn't accept a non-zero host and instead required a developer to use a helper to construct a IPNetwork with a proper address, then there would be less confusion about what exactly a IPNetwork is meant to represent. As it stands, it's purposes is muddled by accepting host addresses too. I am rather indifferent whether there needs to be a IPAddressWithMask type. If that is needed, then it is rather easy to create a type that does that. And, if it is a common pattern, then it could be added to the module later in life. -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

- Masks are also 32- (128-) bit integers, which happen to have the property that their leftmost N bits are all zero and the rest are all one.

As a side note, I would be in favor of dropping the concept of a mask from the library, and only support a prefix length. IPv6 doesn't support masks at all, and even for IPv4, I think there are conventions (if not RFCs) against using them in a way that does not correspond to a prefix length. Regards, Martin

...

As a side note, I would be in favor of dropping the concept of a mask from the library, and only support a prefix length.

-1

...

IPv6 doesn't support masks at all, and even for IPv4, I think there are conventions (if not RFCs) against using them in a way that does not correspond to a prefix length.

Then the module should only support netmasks of the form (say) '255.255.255.224' (equivalent to "/27"), and reject those like "255.3.255.255". It currently accepts them.

Many applications still display netmasks in dot-quad form, and I would be terribly annoyed if I had to count the bits myself before passing it to IPv4Address.

I wouldn't ask for that: it should certainly be possible to supply masks. However, I would want to reject masks that don't correspond to a prefix, and have only the prefix length in the internal representation. Regards, Martin

On Sun, 27 Sep 2009 at 10:06, David Moss wrote:

On 27 Sep 2009, at 07:56, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

I wouldn't ask for that: it should certainly be possible to supply masks. However, I would want to reject masks that don't correspond to a prefix, and have only the prefix length in the internal representation.

+1 on rejection of netmasks without direct CIDR prefix equivalents. AFAIK Cisco routers accept them but I don't see how they would be useful in practice (unless someone can demonstrate their need for this).

Several years ago I ran into a network with a non-CIDR netmask. This was in the early days of CIDR, and I hope that that network is no longer configured that way.... A little googling produced this hit: http://tools.ietf.org/wg/mip4/draft-ietf-mip4-nemo-v4-base/draft-ietf-mip4-n... which while not a standard itself clearly believes that non-contiguous netmasks are valid, and is dated 2008. Earlier RFCs (950 and 1219) give them as valid but discouraged. The linux stack (or at least the ifconfig command on my system) does not appear to support non-contiguous netmasks. From a brief search on Cisco's site it looks like Cisco only partially supports non-contiguous masks (you can use them in wildmasks in access lists, but not in subnet masks on interfaces). (I just tried setting a non-contiguous netmask on an interface on one of my Ciscos, and it gave me a 'bad mask' error message.) I think support for non-contiguous netmasks could be safely omitted, but it would mean the module did not fully support the standards, which should be noted in the docs (and why). --David

...

I argue that we're not actually adding any complexity: yes, we add a class (per protocol), but we then merely relocate functionality to clarify the intended use of the classes.

And I argue the moving this functionality to new classes (and adding new restrictions to existing classes) doesn't buy anything in the way of overall functionality of the module or a developer's ability to comprehend intended uses.

It's mostly just minor refactoring and renaming, which I think makes things clearer, although I agree this is merely an opinion. I would be interest to hear what others think. To summarise: * an address is a singleton (a network endpoint), with no container behaviour. It may optionally reference it's network (via the .network attribute), .address returns mask-less address. * a network is a container-like object. For consistency, .network should return self and raise an exception if the mask conflicts with the address, .address returns the base address, .mask returns an address object.

...

I would argue that a Network never has a single address - by definition, it has two or more nodes. A .network attribute should return a Network instance. If you want the base address, this probably should be called .base_address or just .address (to parallel the .netmask attribute).

.network is shorthand for network address. are .network_address and .broadcast_address less confusing? I have to say, though, .network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4 use, IIRC .network...)

Yes, I understand your motivation, but I still think it's going to be more confusing the way you have it.

...

This isn't about shortcuts, but about correctness... having the Network object represent a network, and having Address objects represent end-points, and having errors discovered as early as possible.

Then what I don't see is the purpose of your network-only-network-object. essentially identical functionality can be obtained with the module as is w/o the added complexity of new classes.

Certainly, I'm not talking about adding functionality. What I am suggesting is that if we wish to have a distinction between networks and addresses, then that distinction should be clear and strong, such that the choice of which to use is obvious, and if the wrong one is used, the error is discovered as early as possible. As the module stands, we have a pair of address-without-mask classes called *Address, and a pair of address-with-mask classes called *Network. So, sometimes when you want to record an *address* you use a class called Network, and that class comes with a behaviours that make no sense in the context of a singleton network end-point (it can't "contain" other addresses, although it's .network can). Sorry if I sound like a cracked record - these are subtle concepts, and my ability to explain what I mean is less than is needed, but we'll get there in the end. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

...

As the module stands, we have a pair of address-without-mask classes called *Address, and a pair of address-with-mask classes called *Network. So, sometimes when you want to record an *address* you use a class called Network, and that class comes with a behaviours that make no sense in the context of a singleton network end-point (it can't "contain" other addresses, although it's .network can).

I'm going to consistently use "address" to mean a singleton and "network" to mean a container in the following.

Ta. I think it's useful to have a common terminology.

I still don't see why an address-with-mask is useful, except that the network is deducible as {'network': address & mask, 'mask': mask}. Is there *any* other way you would *ever* use that?

It seems to me that for some purposes (implementing dig(1), for example), an IPv4Address can contain only the address (ie, a 32-bit integer) as a data attribute, and (with methods for using that attribute) that is the minimal implementation of IPv4Address.

However, there are other cases (eg, routing) where it's useful to associate an address with its network, and I don't see much harm in doing so by adding a 'network' attribute to the base class IPv4Address, since addresses are hardly useful except in the context of networks. Of course that attribute is often going to be None (eg, in implementing dig(1) the remote nameserver is unlikely to tell you the netmask). However, when iterating over an IPv4Network, the iterator can automatically fill in the 'network' attribute, and that's fairly cheap.

Conceptually, you sometimes need a bare address, and other times, you need an address with an associated network (host interface configs, router configs, etc). By AddressWithMask, I really mean AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently, IPv4 and IPv6 addressing allows us to derive the network from the host address combined with the netmask - in other words, we don't have to attach a real Network object to Address objects until the user tries to access it, and then we derive it from the address and mask.

While to me neither the 'network' attribute nor the iterator behavior just described seems amazing useful in the base classes, it seems to me that precisely those behaviors will be reinvented over and over again for derived classes. Furthermore they are natural enough that they won't bother people who don't need them. (That's despite at least one person (IIRC it was Antoine) firmly saying "an IPv4Address should contain exactly one 32-bit int, no more, no less", so I could be wrong.)

If you have a .network attribute on an address object, checking if an address is in the same network as another address becomes: addr_a in addr_b.network As the module stands, you write that as: addr_a in addr_b I don't think the intent is as clear with the later.

It seems to me that the only good reason for not having a 'network' attribute that contains an IPv4Network instance or None is efficiency: the space for the attribute and the overhead of filling it in the iterator. I personally can't think of an application that would care (from what I hear, Cisco has no interest in writing its routers' IP stacks in Python, amazingly enough), but in theory ...

The implementation already lazily creates most things like this.

Finally, I agree that using IPv4Network as address-with-mask is a confusing, undiscoverable abuse. In particular, I think that every time I went a week without using that idiom, I'd get nervous when I saw it again: "Are you *sure* that won't raise an error or silently get the lower bits masked off?! If not now, in the next version?"

Yes. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

...

Conceptually, you sometimes need a bare address, and other times, you need an address with an associated network (host interface configs, router configs, etc). By AddressWithMask, I really mean AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently, IPv4 and IPv6 addressing allows us to derive the network from the host address combined with the netmask - in other words, we don't have to attach a real Network object to Address objects until the user tries to access it, and then we derive it from the address and mask.

To clarify: when you say "derive the network" are you talking about the network (which is a container) or the network address = host_address & netmask (which is not a container)? I think you're referring to the later.

I mean a Network object which is a container (which, by definition, has a network address + mask).

If there's need for address+netmask, does it need to be a separate class? Perhaps Address objects could simply have a netmask property, defaulting to None. If you need an "address with mask" object, you create an Address and set the mask:

addr = Address(...) addr.netmask = "255.255.255.0"

Greg Ewing suggested this yesterday - I'm neutral on whether it's done this way or as a separate class. The implementation may be somewhat cleaner if it's a separate class, however.

...

If you have a .network attribute on an address object, checking if an address is in the same network as another address becomes:

addr_a in addr_b.network

As the module stands, you write that as:

addr_a in addr_b

I don't think the intent is as clear with the later.

I would find the later completely unclear and disturbing -- how can one address contain another address?

Yes - that's how it works now, and I can only see it resulting in confusion and bugs for no advantage. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

2009/9/17 Peter Moody <peter@hda3.com>:

On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara <andrewm@object-craft.com.au> wrote:

...

...

...

I think we're in a painful middle ground now - we should either go back to the idea of a single class (per protocol), or make the distinctions clear (networks are containers and addresses are singletons).

Personally, I think I would be happy with a single class (but I suspect that's just my laziness speaking). However, I think the structure and discipline of three classes (per protocol) may actually make the concepts easier to understand for non-experts.

I think this is where we disagree. I don't think the added complexity does make it any easier to understand.

I argue that we're not actually adding any complexity: yes, we add a class (per protocol), but we then merely relocate functionality to clarify the intended use of the classes.

And I argue the moving this functionality to new classes (and adding new restrictions to existing classes) doesn't buy anything in the way of overall functionality of the module or a developer's ability to comprehend intended uses.

For what it's worth, Andrew's making the distinction clatified things for *me*. Whether I am the target audience for the module (I'm not a network expert, but on the odd occasion I need to manipulate IP addresses in a program, and if the standard library had a module for doing so, I'd reach for it) is up to someone else to decide, but if I am, then my vote is for 3 classes based on the explanations I've seen so far. I don't see what having fewer classes buys me as the end user. Paul.

Another way to approach this would be for the Address object to potentially have a 'network' attribute referencing a Network object.

Yes - that's reasonable.

Then there are only two classes, but three use cases are covered:

1) a Network

2) a plain, network-agnostic Address with network == None

3) an Address with an attached Network

An Address could be constructed in three ways:

Address(ip_number)

Address(ip_number, network = <Network instance>)

Address(ip_number, mask = <mask>) # constructs and attaches a suitably-masked Network instance

I think you still need to support the common notations: Address('10.0.0.1') # .network == None Address('10.0.0.1/255.255.255.0') Address('10.0.0.1/24')

We could also have some_network[n] return an Address referring back to the network object it was obtained from.

Yes. (Of course, we're simplifying - there would really be classes for each protocol). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

Antoine Pitrou wrote:

Le Thu, 17 Sep 2009 14:30:28 +1200, Greg Ewing a écrit :

...

3) an Address with an attached Network

An Address could be constructed in three ways:

Address(ip_number)

Address(ip_number, network = <Network instance>)

Address(ip_number, mask = <mask>) # constructs and attaches a suitably-masked Network instance

We could also have some_network[n] return an Address referring back to the network object it was obtained from.

It seems you are uselessly conflating two perfectly distinct concepts: Address and Network. You also haven't addresses the issue of comparing together (and hashing) two addresses with the same IP but pointing to a different network. No answer to this issue seems satisfactory and obviously right.

As it is, -1 from me. Either we only keep two concepts (Address and Network), or if we introduce a third one (AddressWithMask, whatever) for added practicality; but we shouldn't blur the line between the two former canonical concepts under the pretext that a platypus-like Address might be helpful in some particular situations.

I completely agree with this. By keeping the concepts distinct we can catch mis-uses earlier. If needed, convenience functions (or classes, or whatever) could be layered on top. But the underlying concepts need to be clear. Eric.

Nick Coghlan wrote:

Eric Smith wrote:

...

Antoine Pitrou wrote:

...

As it is, -1 from me. Either we only keep two concepts (Address and Network), or if we introduce a third one (AddressWithMask, whatever) for added practicality; but we shouldn't blur the line between the two former canonical concepts under the pretext that a platypus-like Address might be helpful in some particular situations. I completely agree with this. By keeping the concepts distinct we can catch mis-uses earlier. If needed, convenience functions (or classes, or whatever) could be layered on top. But the underlying concepts need to be clear.

That would be my view as well (this includes getting rid of the current duality of IPNetwork by dropping the ip property, only accepting the network address in the normal constructor and having a separate constructor that allows a network object to be created from an arbitrary host address and a netmask)

Yes, I think that's the best plan. I could live without AddressWithMask (or whatever name), but it would be a nice convenience.

Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly,

LOL! Perfect. Eric.

On Thu, Sep 17, 2009 at 5:32 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:

Eric Smith wrote:

...

Antoine Pitrou wrote:

...

As it is, -1 from me. Either we only keep two concepts (Address and  Network), or if we introduce a third one (AddressWithMask, whatever) for added practicality; but we shouldn't blur the line between the two former canonical concepts under the pretext that a platypus-like Address might be helpful in some particular situations.

I completely agree with this. By keeping the concepts distinct we can  catch mis-uses earlier. If needed, convenience functions (or classes, or whatever) could be layered on top. But the underlying concepts need to be clear.

That would be my view as well (this includes getting rid of the current duality of IPNetwork by dropping the ip property, only accepting the network address in the normal constructor and having a separate constructor that allows a network object to be created from an arbitrary  host address and a netmask)

Again, the same error-catching functionality can be obtained through an option to the constructor. network and broadcast attributes can be renamed to .\1_address to alleviate confusion as well. I mentioned before that IPy's insistence on receiving masked out networks was one of the main reasons I wrote ipaddr to begin with. Having ipaddr mimic this behavior would make it significantly less useful. Removing functionality in the name of avoiding confusion doesn't make sense when the same confusion can be alleviated w/o the loss. off to patch the pep and implement some of the non controversial changes. /peter

Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly, Nick.

-- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia --------------------------------------------------------------- _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Thu, Sep 17, 2009 at 9:26 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

Again, the same error-catching functionality can be obtained through an option to the constructor. network and broadcast attributes can be renamed to .\1_address to alleviate confusion as well.

My remark was not targetting error-catching of non-zero low bits, it was about the suggestion to have a "network" or "mask" attribute on Address objects.

If I'm reading your reply correctly, then I agree that having a network attribute on an address object doesn't make sense. My remark was targeting Eric's reply of, "By keeping the concepts distinct we can catch mis-uses earlier." Cheers, /peter

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Thu, Sep 17, 2009 at 10:32 AM, R. David Murray <rdmurray@bitdance.com> wrote:

On Thu, 17 Sep 2009 at 09:16, Peter Moody wrote:

...

I mentioned before that IPy's insistence on receiving masked out networks was one of the main reasons I wrote ipaddr to begin with. Having ipaddr mimic this behavior would make it significantly less useful. Removing functionality in the name of avoiding confusion doesn't make sense when the same confusion can be alleviated w/o the loss.

We're suggesting moving that functionality (accepting a non-masked IP plus netmask and returning the corresponding Network object) into a different constructor, not eliminating the functionality.

Do you mean a new constructor function, ala IPAddress and IPNetwork (so the current classes remain largely the same and the various constructors enforce certain restrictions) or something else? At this point, I'm getting confused with the intermingling of class and constructor names. Cheers, /peter

--David

R. David Murray wrote:

I would have IPv4Address itself be strict, and thus the new constructors would compute the network address and call the regular IPv4Address constructor.(*)

s/Address/Network/ in this paragraph :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Thu, Sep 17, 2009 at 02:04:11PM -0400, R. David Murray wrote:

I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24').

I'd actually suggest to use

...

...

net, host = parse_network_and_host("192.168.111.33/24") (IPv4Network('192.168.111.0/24'), IPv4Address('192.168.111.33'))

I think this helps the use case of the short network+gateway notation, while keeping the concepts of network and host address cleanly separate.

I would have IPv4Address itself be strict, and thus the new constructors would compute the network address and call the regular IPv4Address constructor.(*)

Then the regular IPv4Network constructor could be strict and can be used for validation. - Sebastian

off to patch the pep and implement some of the non controversial changes.

It might be a good idea to add some use-cases to the PEP. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

...

It might be a good idea to add some use-cases to the PEP.

There are several use-cases in the PEP already.

Maybe the use-cases deserve their own section in the PEP, or better yet, be pulled up into the Motivation section.

The problem is, for every use-case where one can show that the existing implementation is confusing, I can come up with a use-case showing where the existing implementation makes more sense than anything proposed.

Uh, I don't think that is the intention of use-cases - they're there to inform the design, rather than to show how a specific implementation can be used. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

On Thu, 17 Sep 2009 at 20:29, Peter Moody wrote:

On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara <andrewm@object-craft.com.au> wrote:

...

...

off to patch the pep and implement some of the non controversial changes.

It might be a good idea to add some use-cases to the PEP.

There are several use-cases in the PEP already.

The problem is, for every use-case where one can show that the existing implementation is confusing, I can come up with a use-case showing where the existing implementation makes more sense than anything proposed.

Then all of those use cases should be in the PEP, so we can try to find the best refinement we can of the API for handling all of them. --David

Again, the same error-catching functionality can be obtained through an option to the constructor. network and broadcast attributes can be renamed to .\1_address to alleviate confusion as well.

I mentioned before that IPy's insistence on receiving masked out networks was one of the main reasons I wrote ipaddr to begin with. Having ipaddr mimic this behavior would make it significantly less useful. Removing functionality in the name of avoiding confusion doesn't make sense when the same confusion can be alleviated w/o the loss.

The issue is bigger than error checking - I'm maintaining that a distinction between an Address (singleton, item) and a Network (Container) is useful and should be embraced. The current implementation has already partially gone this route, but hasn't completed the transition, and this does not give the structure to users that it could - there's an obligation on modules in the standard library to provide leadership and clarity without being dictatorial. They are essentially silent mentors. So, while I am not suggesting we build a bondage and discipline machine, I am suggesting that partitioning the functionality differently will result in a better outcome all round. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

On Wed, Sep 16, 2009 at 4:59 PM, Greg Ewing <greg.ewing@canterbury.ac.nz>wrote:

Some people have claimed that the gateway address of a network isn't necessarily the zero address in that network.

I'll go further: I don't think it's even legal for the gateway address to be the zero address of the network (and I used to program the embedded software in routers for a living :) ).

If that's true, then you *can't* calculate the network address from a host address and a netmask -- there isn't enough information.

In a router or operating system kernel, an IPv4 address or netmask is stored in a 32-bit unsigned integer. Host addresses, network addresses, and netmasks satisfy the following properties, where & is the bitwise-AND operation: network address & netmask == network address host address & netmask == network address of that host A gateway is just a host that will forward packets. A gateway address has the same properties as a host address and isn't the same as the zero address. Every host has a routing table that contains a list of (network address, gateway, netmask) tuples. To see the list, just run "/sbin/route -n" on any Linux system (and most other Unixes; root is not typically required) or "route print" on a Windows system. To route a packet, the operating system essentially performs the following algorithm: def find_gateway_for_destination(destination_address): for (network_address, gateway, netmask) in list_of_routing_table_entires: if network_address == destination_address & netmask: return gateway raise DestinationUnreachable Furthermore, an IPNetwork object

needs to be able to represent a network address whose address part contains bits that aren't in the mask.

Network addresses never contain bits that aren't in the mask (due to the rule: "network address & netmask == network address"). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

On Thu, 17 Sep 2009 10:41:37 am Andrew McNamara wrote:

In the olden days, the mask was spelled out in octets (eg 255.255.255.0). But we've moved to a more compact and logical notation where the number of leading significant bits is specified (eg /24).

I hope you're not suggesting the older notation be unsupported? I would expect to be able to use a mask like 255.255.255.192 without having to count bits myself. -- Steven D'Aprano

I don't think the RFCs forbid the zero address being used

RFC 1122 does: "IP addresses are not permitted to have the value 0 or -1 for any of the <Host-number>, <Network-number>, or <Subnet- number> fields (except in the special cases listed above)." RFC 3021 modifies this requirement, allowing the zero address to be used for an 31-bit prefix. Regards, Martin

On Sat, Sep 26, 2009 at 2:07 PM, DrKJam <drkjam@gmail.com> wrote:

The current version of the PEP and reference implementation do not mention or deal with IPv4 classful addressing (A, B, C, D and E). It would be good to know if any of this (admittedly older yet no less important) functionality is going to be supported. If the library is to concentrate solely on classless addressing (i.e. CIDR) please can this be stated in future revisions of the PEP.

Classful addressing was deprecated more than 15 years ago! Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such scenarios are no longer operationally relevant." -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com> "The plumage don't enter into it. It's stone dead."

On Sat, Sep 26, 2009 at 4:57 PM, DrKJam <drkjam@gmail.com> wrote:

2009/9/26 Daniel Stutzbach <daniel@stutzbachenterprises.com>

...

On Sat, Sep 26, 2009 at 2:07 PM, DrKJam <drkjam@gmail.com> wrote:

...

The current version of the PEP and reference implementation do not mention or deal with IPv4 classful addressing (A, B, C, D and E). It would be good to know if any of this (admittedly older yet no less important) functionality is going to be supported. If the library is to concentrate solely on classless addressing (i.e. CIDR) please can this be stated in future revisions of the PEP.

Classful addressing was deprecated more than 15 years ago!

Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such scenarios are no longer operationally relevant."

Interesting as evidence of classful IPv4 behaviour seems fairly pervasive in current IP stacks and software that supports IPv4 addressing (e.g. PostgreSQL inet and cidr data types).

PostgreSQL's inet type stores a host address and optionally a network mask. It's cidr type stores a network address and mask. Neither of them are classful.

Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04 install) :- netmask addr

Set the IP network mask for this interface. This value defaults to the usual class A, B or C network mask (as derived from the interface IP address). but it can be set to any value.

Since the network mask can be set to any legal value, that means it's stored as CIDR. It falls back on the classful network mask as a default. Undoubtedly that was useful when ifconfig was updated for CIDR. There were still many classful networks and it needed to retain backward compatibility. After all, ifconfig is around 25 years old, and CIDR is only 16. Today, all IP networks are classless (CIDR), and a system administrator would have to do extra work to figure out if the default value happened to be just right for the network they're setting up.

The point being that you can't always assume /32 implicitly for all use cases.

Certainly, which is why the user can explicitly set the netmask.

Here is how IP addresses without an explicit prefix or netmask are currently handled :-

...

...

...

import ipaddr ipaddr.IPv4Network('10.0.0.1') IPv4Network('10.0.0.1/32')

It may not be something we want to support (you could always force the end user to specify a prefix or netmask explicitly). This is fair enough, but let's indicate that it's a considered choice. Somewhere in the PEP text is might be a good place for treatment of this topic.

+1 on forcing the user to specify a netmask explicitly and documenting it. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

On 16 Sep 2009, at 01:05, Peter Moody wrote:

On Tue, Sep 15, 2009 at 4:34 PM, Scott Dial <scott+python-dev@scottdial.com> wrote:

...

R. David Murray wrote:

...

On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote:

...

Le mardi 15 septembre 2009 à 15:48 -0400, R. David Murray a écrit :

...

However, I do not think that the proposed API should accept, eg, IPv4Network ('192.168.1.1/24') as valid. That's just too confusing and error prone.

Indeed, it should throw some kind of ValueError instead.

Peter, what do you think?

I disagree. It seems overly pedantic to throw a valueerror on this. IPy does (used to do?) this and it's one of the primary reasons I wrote ipaddr.

I've seen user interfaces accept 192.168.1.1/24 as a short cut to set the ipaddr and netmask on an interface. For that use being able to parse that string into an IP Address and a Net Mask is what they want. If this is a common idiom having a utility function that returned an IPv4Address('192.168.1.1') and IPv4Network('192.168.0.0') would be useful. If someone really thinks that '192.168.1.1/16' is a network any good UI should detect it is invalid report that to the user. If you think this is a decision that the only the caller can decide why not add a keyword param to control the parsing? IPv4Network( '192.168.1.1/16', syntax='loose' ) (I'm sure syntax='loose' is not right but you get the idea.) Barry

I again invite interested parties to continue this discussion on ipaddr-py-dev@googlegroups.com. we're pushing 250 messages on PEP 3144 at this point; well beyond what most folks would call a "long open-ended discussion". anyway:

The current behaviour is confusing to me. For example:

...

...

...

netw1 = ipaddr.IPv4Network('192.168.1.1/24') netw2 = ipaddr.IPv4Network('192.168.1.0/24') netw1 == netw2 False list(netw1) == list(netw2) True

I think you're intentionally confusing yourself here. 192.168.1.1 clearly doesn't equal 192.168.1.0, so I don't know why you'd expect 192.168.1.1/24 to equal 192.168.1.0/24.

Two networks, containing exactly the same range of addresses, but they don't compare equal. I'm not convinced that netw1 should even be allowed, but if it is, surely it should be turned into canonical form netw2? E.g. I would expect this:

...

...

...

ipaddr.IPv4Network('192.168.1.1/24') IPv4Network('192.168.1.0/24')

but that's not what it does.

now you're confusing me as before you've asked for this to either: 1. Raise an exception 2. Return two objects. your unconfusing (to you) example does neither one of those.

-- Steven D'Aprano _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Sat, Sep 26, 2009 at 10:38 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

Peter Moody wrote:

...

I again invite interested parties to continue this discussion on ipaddr-py-dev@googlegroups.com.  we're pushing 250 messages on PEP 3144 at this point; well beyond what most folks would call a "long open-ended discussion".

anyway:

...

The current behaviour is confusing to me. For example:

...

...

...

netw1 = ipaddr.IPv4Network('192.168.1.1/24') netw2 = ipaddr.IPv4Network('192.168.1.0/24') netw1 == netw2 False list(netw1) == list(netw2) True

I think you're intentionally confusing yourself here.  192.168.1.1 clearly doesn't equal 192.168.1.0, so I don't know why you'd expect 192.168.1.1/24 to equal 192.168.1.0/24.

No, this is exactly the "dual purpose" nature of IPNetwork that people have been complaining about: 192.168.1.1 & 255.255.255.0 == 192.168.1.0 & 255.255.255.0, so the two objects describe the exact same network, but the network objects don't currently respect their own netmask, instead remembering which particular host happened to be used to derive the network ID.

"Arbitrary network host with associated netmask" and "IP Network" are different concepts, but the library currently uses a single class interface (IPNetwork) for both of them.

What we're asking for is for the "ip" attribute to be removed from IP Network, with the constructor either forgetting the host IP that was used to derive the network address, or else only accepting the correct network address in the constructor and providing a separate helper function or class method to create an IPNetwork object from an arbitrary hostname.

what you want is strict=True, just checked in. I've been meaning to send an update to the PEP explaining why this choice was made. I'm hoping this will ameliorate your confusion. Cheers, /peter

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia ---------------------------------------------------------------

On Sat, Sep 26, 2009 at 11:08 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

Peter Moody wrote:

...

what you want is strict=True, just checked in. I've been meaning to send an update to the PEP explaining why this choice was made. I'm hoping this will ameliorate your confusion.

If the ip property remains and equality is still broken as Steven describes, then no, it won't really alleviate my concerns.

I've also previously registered my objections to using a boolean flag for this instead of a separate method.

However, I'll wait and see what the PEP update says before commenting further.

I've mailed the PEP patch in, hopefully it will be checked in and updated shortly. To be explicit though, unless I'm drastically misreading what you're asking for, I consider the design you're proposing to be broken from a usability perspective (striving for academic/pedantic correctness over accepted practice). I have no intention of adding any sort of AddressWithNetmask classes or making other fundamental design changes you've suggested. If what the community requires is the library you've described, then ipaddr is not that library. Cheers, /peter

Cheers, Nick.

-- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia ---------------------------------------------------------------

Antoine Pitrou writes:

There was a proposal to have a separate parse_address_and_mask method which would return a (Address, Network) tuple, I still don't know why you don't seem to consider it seriously, rather than trying to make the Network class a kind of all-in-one type conflating different concepts.

Because he thinks about the problem space differently from you. Specifically, AFAICS he does not (maybe even "can't") see a reason to distinguish an AddressWithMask from a Network. If so, it's a reasonable API choice for him to choose not to have separate classes to represent such similar concepts. That's all there is to it. I personally do not have a problem with that, except that you apparently can't grasp his way of thinking, and he apparently can't grasp yours. I'm -1 on PEP 3144 primarily because of this communications gap. Secondarily because I agree that it's unnatural that a Network instance can have an arbitrary distinguished address other than those defined in the RFCs (the network and broadcast addresses), especially since it matters to equality comparisons. (But I personally would use ipaddr if it were in the stdlib despite that.)

On Sun, Sep 27, 2009 at 4:23 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

To be explicit though, unless I'm drastically misreading what you're asking for, I consider the design you're proposing to be broken from a usability perspective (striving for academic/pedantic correctness over accepted practice).

It is certainly not academic correctness, it is concern over the understandibility of the library. The behaviour you are proposing is confusing, and even if you think it is more practical (which I doubt it is), it should still be avoided IMO. While "192.168.1.2/24" may be an accepted notation in some situations, it is not a notation for a *network* but for an (address, mask) pair, or an (address, network) pair.

There was a proposal to have a separate parse_address_and_mask method which would return a (Address, Network) tuple, I still don't know why you don't seem to consider it seriously, rather than trying to make the Network class a kind of all-in-one type conflating different concepts.

The reason (aside from the name) that I'm not going to include this in ipaddr is that it would require the user to deal with two objects when one would suffice. It's similar to getting two return values from float(). integer, fraction = float('1.25') crazy, right? Finally, to Stephen's point about seeing the other side of the argument, I wrote this offlist a week ago: I *understand* what you're saying, I *understand* that 192.168.1.1/24 isn't a network, 192.168.1.0/24 is a network, and 192.168.1.1 is an ip address on that network and it has an associated netmask that helps describe its network, etc. etc. what I'm saying is that, enforcing that restriction by introducing 2 new classes (ipv4 is not comparable with ipv6, I've actually got a good quote from Vint Cerf about this, so any AddressWithNetmask (?!?) class would have to be duplicated for both addr types) or something else as of yet described, is not useful. it is in fact, detrimental (optionally enforcing that restriction via an argument to the Network constructors is, however, useful) Cheers, /peter

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Sun, Sep 27, 2009 at 10:22 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

The reason (aside from the name) that I'm not going to include this in ipaddr is that it would require the user to deal with two objects when one would suffice.

You make it sound like it's a burden, but "dealing" with two objects is not something extraordinary or bizarre, it happens all the time when you do network programming: e.g. ('127.0.0.1', 80).

(or would you argue that Address objects should have an optional distinguishing port number, for "convenience" reasons?)

I'm not sure what you're talking about, I've never argued to add layer 4 information to ipaddr.

If you need the two objects, whether you access them through separate variables (`network` and `host`) or through attribute access (`network` and `network.host`) doesn't really matter. You can even pretend the tuple is an atomic object and store it as-is if you want. I'm not sure what annoys you here.

this is tautological. if you need two objects, then two objects is what you need. I'm saying that you don't need two objects for this, that common accepted practice is to represent this as one. Note, there is *nothing* stopping you currently from using two objects for this, an IPAddress object and a strict=True IPNetwork object. I'm not sure what annoys you about this.

...

It's similar to getting two return values from float().

I don't see how it's similar, since a float is a clearly defined atomic entity. Most float operations become meaningless if you consider the integral and the fractional part separately.

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Sun, Sep 27, 2009 at 11:17 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

...

(or would you argue that Address objects should have an optional

distinguishing

...

...

port number, for "convenience" reasons?)

I'm not sure what you're talking about, I've never argued to add layer 4 information to ipaddr.

It was an analogy, just like your "float" analogy.

...

I'm not sure what annoys you about this.

I have already explained it: what annoys me is that it makes `Network` a hybrid object conflating two independent concepts, and makes the library less understandable as a result.

I understand that this is your assertion. Here's the thing, your use case is already supported. IPv?Network objects can only be created of network addresses, exceptions are raised if all the host bits aren't 0, you can create and only deal with IPv4Address('192.168.1.1') and IPv4Network('192.168.1.0/24', strict=True) etc etc. That's not how I (nor many other network administrators) would use it, but it's doable. what you're claiming is that my use case is invalid. that's what I claim is broken. There have been folks on both sides claiming that this design is both fundamentally confusing and fundamentally sound. The confused have obviously been more vocal and less willing to compromise. So, the basic design of the library stands as it is, minor implementation and documentation bugs not withstanding. I'm not going to make ipaddr less useful (strictly removing functionality), more bulky and confusing (adding more confusingly named classes and methods) or otherwise break the library in a vain attempt to have it included in the stdlib. Cheers, /peter

Others have already pointed out that it makes operations like equality confusing.

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight <foom@fuhm.net> wrote:

On Sep 27, 2009, at 3:18 PM, Peter Moody wrote:

...

administrators) would use it, but it's doable. what you're claiming is that my use case is invalid.

that's what I claim is broken.

He's claiming your solution to address your use case is confusing, not that the use case is invalid.

this isn't actually true. Steven D'Aprano wrote: Could you explain what benefit there is for allowing the user to create network objects that don't represent networks? Is there a use-case where these networks-that-aren't-networks are something other than a typo? Under what circumstances would I want to specify a network as 192.168.1.1/24 instead of 192.168.1.0/24? that pretty flatly states that my use case is invalid.

...

I'm not going to make ipaddr less useful (strictly removing functionality), more bulky and confusing (adding more confusingly named classes and methods) or otherwise break the library in a vain attempt to have it included in the stdlib.

If I understand correctly, the proposal for addressing the issue is to make two rather simple changes:

i wish it were this easy.

1) if strict=False, mask off the bits described by the netmask when creating an IPNetwork, such that the host bits are always 0.

I haven't heard anyone suggest auto-masking bits, but otherwise that would be strict=True.

2) add a single new function:

def parse_net_and_addr(s):  return (IPNetwork(s), IPAddress(s.split('/')[0]))

I've only heard talk of new classes and new methods, not new constructor functions. In fact, when I asked for explicit clarification of what was required, a new constructor function "ala IPAddress and IPNetwork (so the current classes remain largely the same and the various constructors enforce certain restrictions)", I was told, no, a new object AddressWithMask was actually required. I've no problem with a constructor like this, but this is not what people have been asking for. Cheers, /peter

James

On Sun, Sep 27, 2009 at 1:15 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight <foom <at> fuhm.net> wrote:

...

On Sep 27, 2009, at 3:18 PM, Peter Moody wrote:

...

administrators) would use it, but it's doable. what you're claiming is that my use case is invalid.

that's what I claim is broken.

He's claiming your solution to address your use case is confusing, not that the use case is invalid.

this isn't actually true.

Steven D'Aprano wrote:

[...]

That's Steven, your original sentence was about me.

my original sentence was about everyone who was arguing that the current implementation was designed for an invalid use-case. it was in reply to your email, but it was to everyone (you, Nick, rdm, Steven, etc)

...

...

1) if strict=False, mask off the bits described by the netmask when creating an IPNetwork, such that the host bits are always 0.

I haven't heard anyone suggest auto-masking bits, but otherwise that would be strict=True.

I would expect strict=True to raise an error if the lower bits are non-zero, not to silently erase them. strict=False would be the option that silently erases lower bits. (that's why it's named `strict`, after all :-))

...

...

2) add a single new function:

def parse_net_and_addr(s):  return (IPNetwork(s), IPAddress(s.split('/')[0]))

I've only heard talk of new classes and new methods, not new constructor functions.

Well, "method" in that context meant "class method" since the results aren't dependent on a particular instance. Of course, both a class method or a module-level function would be fine.

so this is not the response I got when I asked what was required before. Would adding this constructor function satisfy your concerns (given sensible strict settings in the constructor, etc)?

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

...

...

...

def parse_net_and_addr(s):  return (IPNetwork(s), IPAddress(s.split('/')[0]))

I've only heard talk of new classes and new methods, not new constructor functions.

Well, "method" in that context meant "class method" since the results aren't dependent on a particular instance. Of course, both a class method or a module-level function would be fine.

so this is not the response I got when I asked what was required before. Would adding this constructor function satisfy your concerns (given sensible strict settings in the constructor, etc)?

Assuming the Network type loses the notion of a specific host (or host address, or `ip`) attached to it, yes.

this is "less useful (strictly removing functionality)" and is an example of what I explicitly said I was not going to do with ipaddr. Cheers, /peter

Or to put it more clearly:

...

...

...

Network('192.168.0.1/24', strict=False) Network('192.168.0.0/24') Network('192.168.0.1/24', strict=False) == Network('192.168.0.0/24') True

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Sun, Sep 27, 2009 at 3:32 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

this is "less useful (strictly removing functionality)" and is an example of what I explicitly said I was not going to do with ipaddr.

(please note the conditional wording here) Assuming that a significant number of people agree that there is a design problem, if you don't want to make the necessary changes, then I don't see a reason why ipaddr would enter the stdlib.

I've never said otherwise. In fact, from an email last night, "If what the community requires is the library you've described, then ipaddr is not that library." The changes *you* require make ipaddr significantly less useful to me. I'm not prepared to make those changes in an attempt seek acceptance to the stdlib, especially if the stdlib is in such flux that I'll get to do this again in 18 months.

The functionality (IP address handling) hasn't really seen a huge demand.

then no one would be the worse off if I don't make those suggested changes.

On stdlib-sig recently, a number of people complained that our criteria for including existing libraries in the stdlib should be higher (they complained about the quality of some existing modules, including optparse, which by the way prompted the current proposal to get argparse in the stdlib). I think this PEP is a good moment to judge and decide how demanding or tolerant we (and especially the complainers ;-)) want to be.

Sounds like design by committee to satisfy the squeakiest wheel. I'm trying, but I can't think of anything worse. Cheers, /peter

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com

On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:

Peter Moody <peter <at> hda3.com> writes:

...

I've never said otherwise. In fact, from an email last night, "If what the community requires is the library you've described, then ipaddr is not that library." The changes *you* require make ipaddr significantly less useful to me. I'm not prepared to make those changes in an attempt seek acceptance to the stdlib, especially if the stdlib is in such flux that I'll get to do this again in 18 months.

Well, then I'm not sure why we have a PEP at all. If you don't want any significant changes and if you consider it to be *your* library, ipaddr can remain a third-party package that interested people can easily install (no pun ;-)) since AFAIK it's pure Python. It will also make maintenance easier for you, while freeing us (core developers) from having to bother about it in our daily development tasks.

At least that's what I would advocate right now - not sure about what others think.

I think Peter is pretty frustrated by the many attacks on "his" library. There are probably a number of different things going on simultaneous: Peter has been driven into the defense by attacks both reasonable and unreasonable, there have been misunderstandings all around, teasing out use cases (by both parties) has been a problem. Things might have gone differently if the PEP had started out with multiple authors. Maybe it's not too late to add one or more other interested parties to the PEP with the purpose of making the PEP more clearly the result of a consensus-gathering process. Any volunteers? At the same time I don't think a complete reset of the proposed API is necessary. I am rather more thinking of judicious API tweaks in order to cover some new use cases, without requiring a complete rewrite or destroying the usability of the proposal for Peter's original use cases. (In general I am pretty happy with the ipaddr code and API; it looks like what I would have done, but then I am blissfully unaware of some of the issues brought up in this thread.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2009/9/28 Peter Moody <peter@hda3.com>

[cc += david moss]

On Mon, Sep 28, 2009 at 9:39 AM, Guido van Rossum <guido@python.org> wrote:

...

On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:

...

Peter Moody <peter <at> hda3.com> writes:

...

I've never said otherwise. In fact, from an email last night, "If what the community requires is the library you've described, then ipaddr is not that library." The changes *you* require make ipaddr significantly less useful to me. I'm not prepared to make those changes in an attempt seek acceptance to the stdlib, especially if the stdlib is in such flux that I'll get to do this again in 18 months.

Well, then I'm not sure why we have a PEP at all. If you don't want any significant changes and if you consider it to be *your* library, ipaddr can remain a third-party package that interested people can easily install (no pun ;-)) since AFAIK it's pure Python. It will also make maintenance easier for you, while freeing us (core developers) from having to bother about it in our daily development tasks.

At least that's what I would advocate right now - not sure about what others think.

I think Peter is pretty frustrated by the many attacks on "his" library. There are probably a number of different things going on simultaneous: Peter has been driven into the defense by attacks both reasonable and unreasonable, there have been misunderstandings all around, teasing out use cases (by both parties) has been a problem.

Things might have gone differently if the PEP had started out with multiple authors. Maybe it's not too late to add one or more other interested parties to the PEP with the purpose of making the PEP more clearly the result of a consensus-gathering process. Any volunteers?

David called me a little over a week ago and expressed an interest in doing exactly this cross continent/ocean coordination has been a little difficult thus far and I'm not certain what his feelings on this are now.

Sure, I'm happy to volunteer and help out. Let's have a good hard look at all this and see what we can come up with.

...

At the same time I don't think a complete reset of the proposed API is necessary. I am rather more thinking of judicious API tweaks in order to cover some new use cases, without requiring a complete rewrite or destroying the usability of the proposal for Peter's original use cases. (In general I am pretty happy with the ipaddr code and API; it looks like what I would have done, but then I am blissfully unaware of some of the issues brought up in this thread.)

-- --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/> ) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com