Hi, Python's ipaddress module is really good for working with network / IP related thing. I would like to use ipaddress.IPv4/v6Address in creating network interfaces. Like a socket but i can't, the internal things and some standard libraries doesn't support them (even socket). >>> server_address = (ipaddress.IPv4Address('127.0.0.1'), 10000) >>> sock.bind(server_address) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: str, bytes or bytearray expected, not IPv4Address Proposal ======== The best solution we've got[0] is implementing something like PathLike[1] for IP Address objects. For PoC[2] we called it IPLike. It is a protocol (some kind of interface) created with Abstract Base Classes and it forces class to implement __ipaddr__ method (like __fspath__ method). The __ipaddr__ method should return string or bytes formed notation of IP address. It is dotted notation in IPv4, long formed notation in IPv6. The __ipaddr__ method is also an identifier for IPLike classes. IPLike interface's subclasshook checks the __ipaddr__ method of given class when issubclass called. >>> class SomethingRepresentIP: ... def __ipaddr__(self) -> str: ... return '127.0.0.1' ... >>> issubclass(SomethingRepresentIP, IPLike) True >>> isinstance(SomethingRepresentIP(), IPLike) True For PoC[2] we implented __ipaddr__ method to ipaddress._BaseAddress class. It is parent class of IPv4/v6Adress. >>> issubclass(IPv4Address, IPLike) True >>> IPv4Address('127.0.0.1').__ipaddr__() '127.0.0.1' Like os.fspath we need a general-purpose ip-getter. So we implemented ipaddress.get_ipaddr(address) function. When you give it an valid IP like object it returns the value of __ipaddr__ method. >>> get_ipaddr(IPv4Address('127.0.0.1')) '127.0.0.1' When you give it string or bytes or integer object it calls ipaddr.ip_address(address) and re-calls itself with the value it got from ip_address. It helps get_ipaddr to convert integer-formed / byte-formed IPs to string formed IPs with a valid notation. >>> get_ipaddr('127.0.0.1') '127.0.0.1' >>> get_ipaddr(3232235521) '192.168.0.1' >>> get_ipaddr(b'\xC0\xA8\x00\x01') '192.168.0.1' If you give any type else it raises TypeError. Also if you give it a IP like object (an object that has __ipaddr__ method), but the object's __ipaddr__ method returns something that is not string/bytes it raises TypeError. For PoC[2] i added socket to support IP like objects. >>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> server_address = (SomethingRepresentIP(), 10000) >>> sock.bind(server_address) >>> sock.listen(1) >>> conn, client = sock.accept() >>> conn <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 10000), raddr=('127.0.0.1', 49452)> I didn't implement a C API for ipaddress.get_ipaddr. The function is too small for creating a new module, new header and the ipaddress library currently has no C-side module. If needed i can write a function inside the socketmodule.c . Proof-of-Concept (Implementation) ================================= The implementation can be found on github (DamlaAltun/cpython[2]) We have a Abstract Base Class for IP Address protocol called IPLike. (ipaddress.py[3]) class IPLike(abc.ABC): """ Abstract base class for IP (address) like objects. """ @abc.abstractmethod def __ipaddr__(self): """ If it is a IPv4 return in the dotted form, If it is a IPv6 return in the long form. """ raise NotImplementedError @classmethod def __subclasshook__(cls, subclass): return hasattr(subclass, '__ipaddr__') It has a abstract method __ipaddr__ and a subclasshook. The __ipaddr__ method returns ip representation of object. The subclasshook checks for __ipaddr__ method. Then there is a ip-getter called get_ipaddr (ipaddress.py[4]) def get_ipaddr(address): """Return the representation of a ip address like object. If bytes or string or int passed in, get_ipaddr converts it to a IPv4/v6 Address and calls it self again for getting, IP address of IPv4/v6 object. If a IPLike passed it uses IPLike interface to get the value. """ if isinstance(address, (str, bytes, int)): return get_ipaddr(ip_address(address)) addr_type = type(address) try: ip_addr = address.__ipaddr__() except AttributeError: if hasattr(address, '__ipaddr__'): raise else: raise TypeError("expected str, bytes or ipaddress.IPLike object, " "not {}".format(addr_type.__name__)) if isinstance(ip_addr, (str, bytes)): return ip_addr else: raise TypeError("expected {}.__ipaddr__() to return str or bytes, " "not {}".format(addr_type.__name__, type(addr_type).__name__)) It takes one parameter, address. The address might be string, integer, bytes or ipaddress.IPLike If it is string or bytes or integer the get_ipaddr calls ipaddress.ip_address for getting IPv4/v6Address object. The purpose of that is converting bytes/int IP addresses to valid formed string IP addresses. And calls it self again with result of ipaddress.ip_address It tries to get address' __ipaddr__ method, if it can successfully perform it checks the return value of __ipaddr__ method. If return value is string or bytes it returns the ip address. If it is not it raises a TypeError. In the C-Side we can check via _PyObject_LookupSpecial. (socketmodule.c[5]) idna_converter(PyObject *obj, struct maybe_idna *data) { ... _Py_IDENTIFIER(__ipaddr__); ... func = _PyObject_LookupSpecial(obj, &PyId___ipaddr__); if (func != NULL) { obj = PyObject_CallFunctionObjArgs(func, NULL); Py_DECREF(func); } } Thanks for your attention, Damla [0] https://github.com/python/cpython/pull/9956#pullrequestreview-166756568 [1] https://www.python.org/dev/peps/pep-0519 [2] https://github.com/DamlaAltun/cpython/tree/iplike-object [3] https://github.com/DamlaAltun/cpython/blob/iplike-object/Lib/ipaddress.py#L4... [4] https://github.com/DamlaAltun/cpython/blob/iplike-object/Lib/ipaddress.py#L3... [5] https://github.com/DamlaAltun/cpython/blob/iplike-object/Modules/socketmodul...
Should we support bytes? Why? For path name, the bytes are sometimes the only way to encode the file name, Posix operates with bytes internally. But IP addresses are always ASCII strings. I doubt if we shall support bytes for it, complicating the implementation. Support for `int` in `get_addr()` is event more questionable and error-prone. The main consumer of IPLIke API is socket module. Moving IPLike from ipaddress.py into socket.py makes sense for me. Calling `ip_address` from `get_ipaddr()` is another question. I doubt if `get_ipaddr()` is required at all. In my mind `socket.ipaddr(arg)` should be very similar to `os,fspath` and implemented in the same way: 1. Return argument as is if it is a string, 2. Otherwise try to return `type(arg).__ipaddr__(arg)` `socket.ipaddr` should be as fast as possible, don't process extra conversions and have a C Accelerated version used by default. Basically it is a mirror of `os.fspath()`, `socket.IPLike` is a mirror of `os.PathLike`. Your thoughts? On Sun, Oct 21, 2018 at 1:59 PM Damla Altun <initalize.damla@gmail.com> wrote:
Hi,
Python's ipaddress module is really good for working with network / IP related thing. I would like to use ipaddress.IPv4/v6Address in creating network interfaces. Like a socket but i can't, the internal things and some standard libraries doesn't support them (even socket).
>>> server_address = (ipaddress.IPv4Address('127.0.0.1'), 10000) >>> sock.bind(server_address) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: str, bytes or bytearray expected, not IPv4Address
Proposal ========
The best solution we've got[0] is implementing something like PathLike[1] for IP Address objects. For PoC[2] we called it IPLike. It is a protocol (some kind of interface) created with Abstract Base Classes and it forces class to implement __ipaddr__ method (like __fspath__ method). The __ipaddr__ method should return string or bytes formed notation of IP address. It is dotted notation in IPv4, long formed notation in IPv6.
The __ipaddr__ method is also an identifier for IPLike classes. IPLike interface's subclasshook checks the __ipaddr__ method of given class when issubclass called.
>>> class SomethingRepresentIP: ... def __ipaddr__(self) -> str: ... return '127.0.0.1' ... >>> issubclass(SomethingRepresentIP, IPLike) True >>> isinstance(SomethingRepresentIP(), IPLike) True
For PoC[2] we implented __ipaddr__ method to ipaddress._BaseAddress class. It is parent class of IPv4/v6Adress.
>>> issubclass(IPv4Address, IPLike) True >>> IPv4Address('127.0.0.1').__ipaddr__() '127.0.0.1'
Like os.fspath we need a general-purpose ip-getter. So we implemented ipaddress.get_ipaddr(address) function. When you give it an valid IP like object it returns the value of __ipaddr__ method.
>>> get_ipaddr(IPv4Address('127.0.0.1')) '127.0.0.1'
When you give it string or bytes or integer object it calls ipaddr.ip_address(address) and re-calls itself with the value it got from ip_address. It helps get_ipaddr to convert integer-formed / byte-formed IPs to string formed IPs with a valid notation.
>>> get_ipaddr('127.0.0.1') '127.0.0.1' >>> get_ipaddr(3232235521) '192.168.0.1' >>> get_ipaddr(b'\xC0\xA8\x00\x01') '192.168.0.1'
If you give any type else it raises TypeError. Also if you give it a IP like object (an object that has __ipaddr__ method), but the object's __ipaddr__ method returns something that is not string/bytes it raises TypeError.
For PoC[2] i added socket to support IP like objects.
>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> server_address = (SomethingRepresentIP(), 10000) >>> sock.bind(server_address) >>> sock.listen(1) >>> conn, client = sock.accept() >>> conn <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 10000), raddr=('127.0.0.1', 49452)>
I didn't implement a C API for ipaddress.get_ipaddr. The function is too small for creating a new module, new header and the ipaddress library currently has no C-side module. If needed i can write a function inside the socketmodule.c .
Proof-of-Concept (Implementation) =================================
The implementation can be found on github (DamlaAltun/cpython[2]) We have a Abstract Base Class for IP Address protocol called IPLike.
(ipaddress.py[3])
class IPLike(abc.ABC): """ Abstract base class for IP (address) like objects. """
@abc.abstractmethod def __ipaddr__(self): """ If it is a IPv4 return in the dotted form, If it is a IPv6 return in the long form. """ raise NotImplementedError
@classmethod def __subclasshook__(cls, subclass): return hasattr(subclass, '__ipaddr__')
It has a abstract method __ipaddr__ and a subclasshook. The __ipaddr__ method returns ip representation of object. The subclasshook checks for __ipaddr__ method.
Then there is a ip-getter called get_ipaddr
(ipaddress.py[4])
def get_ipaddr(address): """Return the representation of a ip address like object.
If bytes or string or int passed in, get_ipaddr converts it to a IPv4/v6 Address and calls it self again for getting, IP address of IPv4/v6 object. If a IPLike passed it uses IPLike interface to get the value. """
if isinstance(address, (str, bytes, int)): return get_ipaddr(ip_address(address))
addr_type = type(address) try: ip_addr = address.__ipaddr__() except AttributeError: if hasattr(address, '__ipaddr__'): raise else: raise TypeError("expected str, bytes or ipaddress.IPLike object, " "not {}".format(addr_type.__name__))
if isinstance(ip_addr, (str, bytes)): return ip_addr else: raise TypeError("expected {}.__ipaddr__() to return str or bytes, " "not {}".format(addr_type.__name__, type(addr_type).__name__))
It takes one parameter, address. The address might be string, integer, bytes or ipaddress.IPLike If it is string or bytes or integer the get_ipaddr calls ipaddress.ip_address for getting IPv4/v6Address object. The purpose of that is converting bytes/int IP addresses to valid formed string IP addresses. And calls it self again with result of ipaddress.ip_address
It tries to get address' __ipaddr__ method, if it can successfully perform it checks the return value of __ipaddr__ method. If return value is string or bytes it returns the ip address. If it is not it raises a TypeError.
In the C-Side we can check via _PyObject_LookupSpecial.
(socketmodule.c[5])
idna_converter(PyObject *obj, struct maybe_idna *data) { ... _Py_IDENTIFIER(__ipaddr__); ... func = _PyObject_LookupSpecial(obj, &PyId___ipaddr__); if (func != NULL) { obj = PyObject_CallFunctionObjArgs(func, NULL); Py_DECREF(func); } }
Thanks for your attention,
Damla
[0] https://github.com/python/cpython/pull/9956#pullrequestreview-166756568 [1] https://www.python.org/dev/peps/pep-0519 [2] https://github.com/DamlaAltun/cpython/tree/iplike-object [3] https://github.com/DamlaAltun/cpython/blob/iplike-object/Lib/ipaddress.py#L4... [4] https://github.com/DamlaAltun/cpython/blob/iplike-object/Lib/ipaddress.py#L3... [5] https://github.com/DamlaAltun/cpython/blob/iplike-object/Modules/socketmodul...
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Thanks, Andrew Svetlov
I followed your instructions and appended them to the last commit of DamlaAltun/cpython (branch: _iplike-socket)[0] .
- Bytes and Integer support removed, only allowed string and IPLike. If string passed in it returns without any func call / modification. [1] - IPLike moved to socket [2] - `get_ipaddr()` renamed as `_ipaddr` and moved to socket. [3] - IPLike & ipaddr() tests moved to test_socket [4] - Created a header (ipaddrmodule.h) file and imported into Python.h and added header files list in makefile.pre.in [5] - In socketmodule.c; - Created `PySocket_IPAddr(PyObject *address)[6], its behavior exactly same with socket._ipaddr's behavior. In Lib/socket.py it checks builtins for ipaddr method if it can not find it it renames _ipaddr as ipaddr [7]. - Created `socket_ipaddr_impl(PyObject *module, PyObject *address) [8], it used for exporting c method to python [9]. It calls PySocket_IPAddr. - Added IPLike objects support to socket's idna_converter [10]
[0] https://github.com/DamlaAltun/cpython/tree/_iplike-socket [1] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [2] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [3] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [4] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [5] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [6] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [7] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [8] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [9] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94... [10] https://github.com/DamlaAltun/cpython/commit/d39a261edb91d6e423d319b1dd20f94...
participants (2)
-
Andrew Svetlov -
Damla Altun