macaddress or networkaddress module

Currently in Python 3 we have an ipaddress module which allows easy manipulation of IP version 4 and 6 addresses and networks. Examples of this are getting the packed bytes representation of an address, checking if an address is of a specific type according to the RFC (private, multicast, etc.) My proposal is that we have something with a similar API for MAC addresses. A lot of these operations that we have for IPv4 and IPv6 addresses are also applicable for MAC addresses. We can take in concatenated, hyphenated or dotted notations. Some of the methods could include packed is_multicast is_broadcast is_universal get_oui and optionally format outputs dotted(upper=False) hyphenated(upper=False) concatenated(upper=False) get_oui could initially just return the OUI part of the address without resolving. However, it is possible that on build of Python we pull down the latest OUIs from IEEE into a python file. My initial thought if people are in agreement was to have a macaddress.py module. However if the APIs for these are similar enough, perhaps it warrants having a networkaddress.py module that contains both the old ipaddress code and the new macaddress code? I would be grateful to hear people's thoughts on this. Pete Wicken

This is a perfect idea for a third party package to be put on PyPi. Have you looked to see if one exists already? If, at some point, there is a robust and mature version that is widely used, then it might be worth considering g for the standard library. -CHB On Sat, Apr 4, 2020 at 4:53 AM Pete Wicken <petewicken@gmail.com> wrote:
Currently in Python 3 we have an ipaddress module which allows easy manipulation of IP version 4 and 6 addresses and networks. Examples of this are getting the packed bytes representation of an address, checking if an address is of a specific type according to the RFC (private, multicast, etc.)
My proposal is that we have something with a similar API for MAC addresses. A lot of these operations that we have for IPv4 and IPv6 addresses are also applicable for MAC addresses. We can take in concatenated, hyphenated or dotted notations. Some of the methods could include packed is_multicast is_broadcast is_universal get_oui
and optionally format outputs
dotted(upper=False) hyphenated(upper=False) concatenated(upper=False)
get_oui could initially just return the OUI part of the address without resolving. However, it is possible that on build of Python we pull down the latest OUIs from IEEE into a python file.
My initial thought if people are in agreement was to have a macaddress.py module. However if the APIs for these are similar enough, perhaps it warrants having a networkaddress.py module that contains both the old ipaddress code and the new macaddress code?
I would be grateful to hear people's thoughts on this.
Pete Wicken _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GRCNHO... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

Agreed that this could be achieved by a third party library; and probably already is - but the ipaddress module in Python could also have been one. Therefore I feel it would somewhat complete the already existing networking utilities already included if we were to implement this within the standard library. It is for this reason i also recommend an API that is familiar to people who already use the ipaddress module; as this would be a layer 2 compliment to the existing package. Happy to discuss, Pete Wicken On Sat, Apr 4, 2020 at 8:14 PM Christopher Barker <pythonchb@gmail.com> wrote:
This is a perfect idea for a third party package to be put on PyPi. Have you looked to see if one exists already?
If, at some point, there is a robust and mature version that is widely used, then it might be worth considering g for the standard library.
-CHB
On Sat, Apr 4, 2020 at 4:53 AM Pete Wicken <petewicken@gmail.com> wrote:
Currently in Python 3 we have an ipaddress module which allows easy manipulation of IP version 4 and 6 addresses and networks. Examples of this are getting the packed bytes representation of an address, checking if an address is of a specific type according to the RFC (private, multicast, etc.)
My proposal is that we have something with a similar API for MAC addresses. A lot of these operations that we have for IPv4 and IPv6 addresses are also applicable for MAC addresses. We can take in concatenated, hyphenated or dotted notations. Some of the methods could include packed is_multicast is_broadcast is_universal get_oui
and optionally format outputs
dotted(upper=False) hyphenated(upper=False) concatenated(upper=False)
get_oui could initially just return the OUI part of the address without resolving. However, it is possible that on build of Python we pull down the latest OUIs from IEEE into a python file.
My initial thought if people are in agreement was to have a macaddress.py module. However if the APIs for these are similar enough, perhaps it warrants having a networkaddress.py module that contains both the old ipaddress code and the new macaddress code?
I would be grateful to hear people's thoughts on this.
Pete Wicken _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GRCNHO... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Apr 4, 2020, at 12:45, Pete Wicken <petewicken@gmail.com> wrote:
Agreed that this could be achieved by a third party library; and probably already is - but the ipaddress module in Python could also have been one.
Well, it was. Go read PEP 3144 that added it, and the discussion threads behind it. For years, there were attempts to propose an IP address library to Python that didn’t even make it as far as the PEP stage. Finally someone proposed adopting Google’s ipaddr—which wasn’t quite a category killer, but was very popular, and had a stable API, and the authors were willing to donate it. And it was rejected. Peter Moody took all the issues that came of during that discussion and forked ipaddr and came up with a new design that answered all of people’s concerns. He published a reference implementation and for months took feedback on it and improved it. Then he was ready to propose that for provisional acceptance into the stdlib (and to convert the external package into a backport for older versions of Python if it was accepted). And, after another round of bikeshedding, it got in. The existence of ipaddress should make the process for macaddress easier. (For one thing, a lot of design questions where two options seem good can be answered quickly by saying “ipaddress has the same question, see this thread, and they decided on A, so netaddress should also decide on A, both for the same reasons and for compatibility with ipaddress”.) But it still has to be done. If there already is a good library on PyPI today, it would make more sense to adopt it, or to use it as a starting point, than to start from scratch. The same way ipaddress happened—but hopefully less work. If there isn’t a good existing candidate, so it has to be designed from scratch, very few things go directly into the stdlib. Even things that are designed from the start to land there, like tulip (designed to become asyncio, which it did) and regex (designed to replace re, which it still hasn’t after 8 years or work) are almost always shared and used in the field before they’re proposed for inclusion (and then converted into a backport after inclusion). Although again, the process should be a lot easier with macaddress than with something as complicated as regex or as unprecedented as tulip.

On Apr 4, 2020, at 12:45, Pete Wicken <petewicken@gmail.com> wrote:
Agreed that this could be achieved by a third party library; and probably already is
PS, I’ve needed something like this a few times in the past, so I can offer some vague memories of what’s available. I’ve mostly used netifaces, which is amazing and does all the low-level stuff you’d want and is more cross-platform than you’d believe possible. But its API isn’t very ipaddress-like; it’s mostly dicts of strings and enum ints, and most of what it does isn’t relevant here. There’s a library that I believe is called netaddr (and I think it’s the one with that name on PyPI), which took Google’s ipaddr and expanded it to add MAC/EUI and various other kinds of addresses. If the author is willing, and if it’s solid enough, I’d bet that stripping it down to just MAC and changing the API to parallel ipaddress instead of ipaddr would be a lot easier. I’ve also used QtNetwork in a GUI app. That obviously isn’t as amenable to borrowing, but if there are any design questions that come up, looking at how Qt does it (and whether they have a billion bug reports/feature requests complaining about it) can be helpful.

Andrew, Thanks for the information on the history of the ipaddress module - the background reading is very useful. With regards to using existing Pypi modules, that does sounds like a very sensible approach if we can find something suitable; I will research some of the modules you mentioned as well as some others and see if they could be adapted. I guess my main objective here is to see if there is sufficient interest from others to warrant pursuing the idea. Personally I have found that stdlib support for networking is strangely patchy at the moment; with ipaddress being a great handy module for IP work without requiring dependencies; but seemingly very little networking support before or after that. The 'before' part is what my idea is attempting to tackle. I am not recommending we start investigating further layers of the networking stack post-IP, however I feel the logical step forwards is to have some form of support for the most common form of layer 2 addressing. Given what i have looked at from your recommendations so far, netaddr seems to be the most fitting at the moment. Of course an argument can be made that people should simply look to modules like netaddr for their work, however given the point in the above paragraph and that installing a third party module may feel overkill for certain environments that would typically run code performing mac and ip address inspection (e.g. scripts/small programs on a network switch), it is my personal opinion that this would be well suited in the stdlib (which I recognise would require a lot of work and scrutiny to get there). I'm looking forward to hearing people's opinions on this. On Sat, Apr 4, 2020 at 10:01 PM Andrew Barnert <abarnert@yahoo.com> wrote:
On Apr 4, 2020, at 12:45, Pete Wicken <petewicken@gmail.com> wrote:
Agreed that this could be achieved by a third party library; and
probably already is
PS, I’ve needed something like this a few times in the past, so I can offer some vague memories of what’s available.
I’ve mostly used netifaces, which is amazing and does all the low-level stuff you’d want and is more cross-platform than you’d believe possible. But its API isn’t very ipaddress-like; it’s mostly dicts of strings and enum ints, and most of what it does isn’t relevant here.
There’s a library that I believe is called netaddr (and I think it’s the one with that name on PyPI), which took Google’s ipaddr and expanded it to add MAC/EUI and various other kinds of addresses. If the author is willing, and if it’s solid enough, I’d bet that stripping it down to just MAC and changing the API to parallel ipaddress instead of ipaddr would be a lot easier.
I’ve also used QtNetwork in a GUI app. That obviously isn’t as amenable to borrowing, but if there are any design questions that come up, looking at how Qt does it (and whether they have a billion bug reports/feature requests complaining about it) can be helpful.

On Apr 5, 2020, at 02:48, Pete Wicken <petewicken@gmail.com> wrote:
Personally I have found that stdlib support for networking is strangely patchy at the moment; with ipaddress being a great handy module for IP work without requiring dependencies; but seemingly very little networking support before or after that. The 'before' part is what my idea is attempting to tackle. I am not recommending we start investigating further layers of the networking stack post-IP, however I feel the logical step forwards is to have some form of support for the most common form of layer 2 addressing.
I think Python is just following history here. It has nice wrappers for everything C/POSIX/Sockets exposed, and then it has higher-level libraries for all the protocols that were really important in the early 90s, and that’s it. And then it has selectors and asyncio, and above or below that level to replace all the old stuff. But if the stdlib were designed from scratch today rather than over the past 30 years, I think it would have less than it does, not more. Up to around a decade ago, installing third-party libraries was a huge mess, but nowadays, PyPI works, Python comes with pip pre-installed, and telling developers they need internet access when they first start a new project isn’t considered onerous. And meanwhile, a lot of things people need in the real world evolve a lot faster than Python can keep up with. So the Python documentation now points people to third-party libs like requests and numpy, and the trend is more in that direction rather than the stdlib trying to do everything. (Even pip uses requests rather than anyone trying to update urllib and cert management and so on to make it unnecessary.) However, I think that a macaddress library could easily be one of the few things that does properly fit in the networking section of the stdlib. It’s a small feature, and it should be very stable—I think the last change to the protocol spec was about 20 years ago, and from a brief scan of the release notes for netaddr it looks like for the last decade its needed less than one bugfix or feature per year for dealing with MAC/EUI addresses…

Andrew Barnert via Python-ideas writes:
However, I think that a macaddress library could easily be one of the few things that does properly fit in the networking section of the stdlib. It’s a small feature, and it should be very stable—I think the last change to the protocol spec was about 20 years ago, and from a brief scan of the release notes for netaddr it looks like for the last decade its needed less than one bugfix or feature per year for dealing with MAC/EUI addresses…
I wonder if it should go in a package of its own, or if we should expand ipaddress to netaddress and have all the features at the same level, or if we should have ipaddress.macaddress (the semantics there are a bit wonky, but "ipaddress" is now canonical for the ipaddress features). From the Old Farts Must Reminisce Department:
But if the stdlib were designed from scratch today rather than over the past 30 years, I think it would have less than it does, not more. Up to around a decade ago, installing third-party libraries was a huge mess, but nowadays, PyPI works, Python comes with pip pre-installed, and telling developers they need internet access when they first start a new project isn’t considered onerous.
That's OK for developers on their "own" machines, but I think there are still a lot of folks who need approval from corporate (though I haven't heard from Paul Moore on that theme in quite a while, maybe it's gotten better?) But for those not constrained by corporate policy, even if they don't have good network access consistently, installing everything you think you might ever need from PyPI on the infrequent occasions you do have such access is only a small fraction of modern disks. Suboptimal, of course, but there was a time I typed the bootstrap assembly code for a C compiler from a book, and then did the optimized full-featured self-hosting version the same way. Not saying "you kidz these dayz don't know how good you have it", rather "where there's a will there's a way." $python -m pip + PyPI makes TOOWTDI for packages quite painless! Steve

On Apr 5, 2020, at 20:12, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Andrew Barnert via Python-ideas writes:
However, I think that a macaddress library could easily be one of the few things that does properly fit in the networking section of the stdlib. It’s a small feature, and it should be very stable—I think the last change to the protocol spec was about 20 years ago, and from a brief scan of the release notes for netaddr it looks like for the last decade its needed less than one bugfix or feature per year for dealing with MAC/EUI addresses…
I wonder if it should go in a package of its own, or if we should expand ipaddress to netaddress and have all the features at the same level, or if we should have ipaddress.macaddress (the semantics there are a bit wonky, but "ipaddress" is now canonical for the ipaddress features).
If I were a sysadmin who didn’t know Python well, I’m pretty sure I wouldn’t go looking in a module named ipaddress for functions and classes to deal with MAC addresses. But on the other hand, if I went on StackOverflow or ServerFault and learned that’s where they were. I don’t think I’d complain. So… maybe we need a survey of what other libraries across languages do to guide us in the naming? The other big question is whether to use the “correct” names, so an eui_address function gives you an EUI48Address (or EUI64Address as appropriate), or the names real sysadmins will expect (unless they’re dealing with Bluetooth or FireWire or something instead of Ethernet or WiFi), so you just call mac_address and get a MACAddress.
From the Old Farts Must Reminisce Department:
But if the stdlib were designed from scratch today rather than over the past 30 years, I think it would have less than it does, not more. Up to around a decade ago, installing third-party libraries was a huge mess, but nowadays, PyPI works, Python comes with pip pre-installed, and telling developers they need internet access when they first start a new project isn’t considered onerous.
That's OK for developers on their "own" machines, but I think there are still a lot of folks who need approval from corporate
Sure, but in this particular case, the people writing maintenance scripts that need to deal with MAC addresses are probably the IT department. They get to make the policy and/or administer the intranet PyPI repo, and somehow that always seems to make things easier for them than for the engineering department. :)
But for those not constrained by corporate policy, even if they don't have good network access consistently, installing everything you think you might ever need from PyPI on the infrequent occasions you do have such access is only a small fraction of modern disks. Suboptimal, of course, but there was a time I typed the bootstrap assembly code for a C compiler from a book, and then did the optimized full-featured self-hosting version the same way. Not saying "you kidz these dayz don't know how good you have it", rather "where there's a will there's a way." $python -m pip + PyPI makes TOOWTDI for packages quite painless!
A few months ago, I was on a plane, less than an hour away from landing and being online, and I wanted to pip install something into Pythonista on my iPad and the wait drove me crazy. And then I remembered an article in Cider with 6502 code for playing polyphonic (well, duophonic) audio and getting to the last page and seeing “code continues next issue” and having to wait a month to get the rest of it, and suddenly 40 minutes didn’t seem… well, no, it was still annoying as hell, I was just amazed at how patient that younger me must have been. 300 baud modems, overnight compiles, having to go to the library on the weekend to look up something simple instead of just checking on your phone… But we were happy in them days, though we had no bandwidth. You try and tell t’young people today. And they won’t believe ye.

On Mon, 6 Apr 2020 at 04:14, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
That's OK for developers on their "own" machines, but I think there are still a lot of folks who need approval from corporate (though I haven't heard from Paul Moore on that theme in quite a while, maybe it's gotten better?)
Howdy :-) It's actually got a bit worse, to the point where I don't bother trying to fight the system as much any more, so things have gone quieter because of that :-) But yes, I still believe that there are important use cases for code that only uses the stdlib, and "not being able to get to the internet" is a valid use case that we shouldn't just dismiss. On the other hand, with regard to the comment you were replying to:
Up to around a decade ago, installing third-party libraries was a huge mess, but nowadays, PyPI works, Python comes with pip pre-installed, and telling developers they need internet access when they first start a new project isn’t considered onerous.
I think all of that is true. What I think is more important, though, is around things that aren't so much "projects" as "scripts", and that *distributing* such scripts that depend on 3rd party libraries is still problematic. Here, I'm not talking about making full-fledged packages, or py2exe style fully bundled apps, but "here's this monitoring script I wrote, let's ship it across 10 production systems and why don't you try it out on your dev boxes?" There's a *significant* step change in complexity between doing that if the script depends on just the stdlib, versus if it depends on a 3rd party module. And we don't have a really good answer to that yet (zipapps, and tools like shiv that wrap zipapps, are pretty good in practice, but they are not well known or commonly used). Paul
participants (5)
-
Andrew Barnert
-
Christopher Barker
-
Paul Moore
-
Pete Wicken
-
Stephen J. Turnbull