Where to put non-collection ABCs (was: Deprecating the old-style sequence protocol)
Speaking of using ABCs more, where should we put ABCs which have nothing to do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__. I opened http://bugs.python.org/issue25637 to discuss this, but I figured a wider discussion wouldn't hurt. Some suggest just putting the ABCs into the abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there. Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc. On Sun, Dec 27, 2015, 09:35 Guido van Rossum <guido@python.org> wrote:
I think there's a lot of interesting stuff in this thread. Personally I don't think we should strive to distinguish between mappings and sequences structurally. We should instead continue to encourage inheriting from (or registering with) the corresponding ABCs. The goal is to ensure that there's one best-practice way to distinguish mappings from sequences, and it's by using isinstance(x, Sequence) or isinstance(x, Mapping).
If we want some way to turn something that just defines __getitem__ and __len__ into a proper sequence, it should just be made to inherit from Sequence, which supplies the default __iter__ and __reversed__. (Registration is *not* good enough here.) If we really want a way to turn something that just supports __getitem__ into an Iterable maybe we can provide an additional ABC for that purpose; let's call it a HalfSequence until we've come up with a better name. (We can't use Iterable for this because Iterable should not reference __getitem__.)
I also think it's fine to introduce Reversible as another ABC and carefully fit it into the existing hierarchy. It should be a one-trick pony and be another base class for Sequence; it should not have a default implementation. (But this has been beaten to death in other threads -- it's time to just file an issue with a patch.)
-- --Guido van Rossum (python.org/~guido)
_______________________________________________
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 28.12.2015 18:42, Brett Cannon wrote:
Speaking of using ABCs more, where should we put ABCs which have nothing to do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__.
I opened http://bugs.python.org/issue25637 to discuss this, but I figured a wider discussion wouldn't hurt. Some suggest just putting the ABCs into the abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there.
Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc.
I'd put them into the abc module (perhaps turning this into a package, if things get too crowded). collections.abc could then do a "from abc import *" for b/w compatibility.
On Sun, Dec 27, 2015, 09:35 Guido van Rossum <guido@python.org> wrote:
I think there's a lot of interesting stuff in this thread. Personally I don't think we should strive to distinguish between mappings and sequences structurally. We should instead continue to encourage inheriting from (or registering with) the corresponding ABCs. The goal is to ensure that there's one best-practice way to distinguish mappings from sequences, and it's by using isinstance(x, Sequence) or isinstance(x, Mapping).
If we want some way to turn something that just defines __getitem__ and __len__ into a proper sequence, it should just be made to inherit from Sequence, which supplies the default __iter__ and __reversed__. (Registration is *not* good enough here.) If we really want a way to turn something that just supports __getitem__ into an Iterable maybe we can provide an additional ABC for that purpose; let's call it a HalfSequence until we've come up with a better name. (We can't use Iterable for this because Iterable should not reference __getitem__.)
I also think it's fine to introduce Reversible as another ABC and carefully fit it into the existing hierarchy. It should be a one-trick pony and be another base class for Sequence; it should not have a default implementation. (But this has been beaten to death in other threads -- it's time to just file an issue with a patch.)
-- --Guido van Rossum (python.org/~guido)
_______________________________________________
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Dec 28 2015)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On 29 December 2015 at 03:58, M.-A. Lemburg <mal@egenix.com> wrote:
On 28.12.2015 18:42, Brett Cannon wrote:
Speaking of using ABCs more, where should we put ABCs which have nothing to do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__.
I opened http://bugs.python.org/issue25637 to discuss this, but I figured a wider discussion wouldn't hurt. Some suggest just putting the ABCs into the abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there.
Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc.
I'd put them into the abc module (perhaps turning this into a package, if things get too crowded).
collections.abc could then do a "from abc import *" for b/w compatibility.
With the benefit of hindsight, I think a broad namespace separation like that might have been a good way to go from a data model discoverability perspective, as it clearly separates the abstract descriptions of data and control flow modelling concepts from the concrete implementations of those concepts. However, we should also keep in mind which standard library modules already publish ABCs, which is at least: typing io numbers collections.abc selectors email (That list was collected by grepping Python files for ABCMeta, so I may have missed some) That suggests to me that this design decision has effectively already been made, and it's to include the ABCs in the relevant domain specific modules. The inclusion of the non-collections related ABCs in collections.abc are the anomaly, which can be addressed by moving them out to a more appropriate location (adjusting the documentation accordingly), and then importing them into collections.abc for backwards compatibility (taking care not to increase the startup import footprint too much in the process). The current collections.abc interfaces which I think are most at issue here: Callable Iterable Iterator Generator Awaitable Coroutine Awaitable AsyncIterable AsyncIterator (I'm excluding Hashable from the list, as the main reason that matters is in describing whether or not something is suitable for inclusion in a dict or set) These differ from the rest of the collections.abc interfaces in that they're more closely associated with language level control flow syntax than they are with containers specifically: Callable - function calls Iterable - for loops, comprehensions Iterator - for loops, comprehensions Generator - generators, generator expressions Awaitable - await expressions Coroutine - async def AsyncIterable - async for AsyncIterator - async for Adding ContextManager and AsyncContextManager would give ABCs for the protocols related to "with" and "async with". Since these all correspond to syntactic protocols, I now think it's reasonable to include them directly in the "abc" namespace, since that still gives us a clear guideline for which ABCs go there, and which belong somewhere else: if it has syntax associated with it, or it's part of the ABC machinery itself, then it can go directly in the "abc" namespace. Using the "abc" namespace also ensures there isn't any additional import overhead, since that gets imported by all ABC using code anyway. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
With MAL, Nick, and Yury all suggesting the syntax-related ABCs go into the abc module and no one else really suggesting otherwise beyond me, I think it's reasonable to go with the syntax rule-of-thumb for what ABCs go into abc and all other abstract concepts go into domain-specific modules (e.g., import-related ones go into importlib.abc, which Nick forgot to list :). On Mon, 28 Dec 2015 at 19:58 Nick Coghlan <ncoghlan@gmail.com> wrote:
On 28.12.2015 18:42, Brett Cannon wrote:
Speaking of using ABCs more, where should we put ABCs which have nothing to do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__.
I opened http://bugs.python.org/issue25637 to discuss this, but I
wider discussion wouldn't hurt. Some suggest just putting the ABCs into
On 29 December 2015 at 03:58, M.-A. Lemburg <mal@egenix.com> wrote: figured a the
abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there.
Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc.
I'd put them into the abc module (perhaps turning this into a package, if things get too crowded).
collections.abc could then do a "from abc import *" for b/w compatibility.
With the benefit of hindsight, I think a broad namespace separation like that might have been a good way to go from a data model discoverability perspective, as it clearly separates the abstract descriptions of data and control flow modelling concepts from the concrete implementations of those concepts.
However, we should also keep in mind which standard library modules already publish ABCs, which is at least:
typing io numbers collections.abc selectors email
(That list was collected by grepping Python files for ABCMeta, so I may have missed some)
That suggests to me that this design decision has effectively already been made, and it's to include the ABCs in the relevant domain specific modules. The inclusion of the non-collections related ABCs in collections.abc are the anomaly, which can be addressed by moving them out to a more appropriate location (adjusting the documentation accordingly), and then importing them into collections.abc for backwards compatibility (taking care not to increase the startup import footprint too much in the process).
The current collections.abc interfaces which I think are most at issue here:
Callable Iterable Iterator Generator Awaitable Coroutine Awaitable AsyncIterable AsyncIterator
(I'm excluding Hashable from the list, as the main reason that matters is in describing whether or not something is suitable for inclusion in a dict or set)
These differ from the rest of the collections.abc interfaces in that they're more closely associated with language level control flow syntax than they are with containers specifically:
Callable - function calls Iterable - for loops, comprehensions Iterator - for loops, comprehensions Generator - generators, generator expressions Awaitable - await expressions Coroutine - async def AsyncIterable - async for AsyncIterator - async for
Adding ContextManager and AsyncContextManager would give ABCs for the protocols related to "with" and "async with".
Since these all correspond to syntactic protocols, I now think it's reasonable to include them directly in the "abc" namespace, since that still gives us a clear guideline for which ABCs go there, and which belong somewhere else: if it has syntax associated with it, or it's part of the ABC machinery itself, then it can go directly in the "abc" namespace. Using the "abc" namespace also ensures there isn't any additional import overhead, since that gets imported by all ABC using code anyway.
Regards, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Would it be a good idea to mix 'concrete implementations of ABCs'* directly in the abc module where the tooling to create ABCs live, or to put it in a submodule? I feel it should be a submodule, but that isn't based on vast experience. * Yes, I know, these are not concrete implementations of types... I find it a little confusing to describe. On 12/28/2015 22:58, Nick Coghlan wrote:
Speaking of using ABCs more, where should we put ABCs which have nothing to do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__.
I opened http://bugs.python.org/issue25637 to discuss this, but I figured a wider discussion wouldn't hurt. Some suggest just putting the ABCs into the abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there.
Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc. I'd put them into the abc module (perhaps turning this into a
On 28.12.2015 18:42, Brett Cannon wrote: package, if things get too crowded).
collections.abc could then do a "from abc import *" for b/w compatibility. With the benefit of hindsight, I think a broad namespace separation
On 29 December 2015 at 03:58, M.-A. Lemburg <mal@egenix.com> wrote: like that might have been a good way to go from a data model discoverability perspective, as it clearly separates the abstract descriptions of data and control flow modelling concepts from the concrete implementations of those concepts.
However, we should also keep in mind which standard library modules already publish ABCs, which is at least:
typing io numbers collections.abc selectors email
(That list was collected by grepping Python files for ABCMeta, so I may have missed some)
That suggests to me that this design decision has effectively already been made, and it's to include the ABCs in the relevant domain specific modules. The inclusion of the non-collections related ABCs in collections.abc are the anomaly, which can be addressed by moving them out to a more appropriate location (adjusting the documentation accordingly), and then importing them into collections.abc for backwards compatibility (taking care not to increase the startup import footprint too much in the process).
The current collections.abc interfaces which I think are most at issue here:
Callable Iterable Iterator Generator Awaitable Coroutine Awaitable AsyncIterable AsyncIterator
(I'm excluding Hashable from the list, as the main reason that matters is in describing whether or not something is suitable for inclusion in a dict or set)
These differ from the rest of the collections.abc interfaces in that they're more closely associated with language level control flow syntax than they are with containers specifically:
Callable - function calls Iterable - for loops, comprehensions Iterator - for loops, comprehensions Generator - generators, generator expressions Awaitable - await expressions Coroutine - async def AsyncIterable - async for AsyncIterator - async for
Adding ContextManager and AsyncContextManager would give ABCs for the protocols related to "with" and "async with".
Since these all correspond to syntactic protocols, I now think it's reasonable to include them directly in the "abc" namespace, since that still gives us a clear guideline for which ABCs go there, and which belong somewhere else: if it has syntax associated with it, or it's part of the ABC machinery itself, then it can go directly in the "abc" namespace. Using the "abc" namespace also ensures there isn't any additional import overhead, since that gets imported by all ABC using code anyway.
Regards, Nick.
On Thu, Dec 31, 2015, 07:08 Alexander Walters <tritium-list@sdamon.com> wrote:
Would it be a good idea to mix 'concrete implementations of ABCs'* directly in the abc module where the tooling to create ABCs live, or to put it in a submodule? I feel it should be a submodule, but that isn't based on vast experience.
* Yes, I know, these are not concrete implementations of types... I find it a little confusing to describe.
It's a possibility. Feel free to comment on the issue if you want to discuss this further. -Brett
On 29 December 2015 at 03:58, M.-A. Lemburg <mal@egenix.com> wrote:
On 28.12.2015 18:42, Brett Cannon wrote:
Speaking of using ABCs more, where should we put ABCs which have nothing to do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__.
I opened http://bugs.python.org/issue25637 to discuss this, but I
On 12/28/2015 22:58, Nick Coghlan wrote: figured a
wider discussion wouldn't hurt. Some suggest just putting the ABCs into the abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there.
Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc. I'd put them into the abc module (perhaps turning this into a package, if things get too crowded).
collections.abc could then do a "from abc import *" for b/w compatibility. With the benefit of hindsight, I think a broad namespace separation like that might have been a good way to go from a data model discoverability perspective, as it clearly separates the abstract descriptions of data and control flow modelling concepts from the concrete implementations of those concepts.
However, we should also keep in mind which standard library modules already publish ABCs, which is at least:
typing io numbers collections.abc selectors email
(That list was collected by grepping Python files for ABCMeta, so I may have missed some)
That suggests to me that this design decision has effectively already been made, and it's to include the ABCs in the relevant domain specific modules. The inclusion of the non-collections related ABCs in collections.abc are the anomaly, which can be addressed by moving them out to a more appropriate location (adjusting the documentation accordingly), and then importing them into collections.abc for backwards compatibility (taking care not to increase the startup import footprint too much in the process).
The current collections.abc interfaces which I think are most at issue here:
Callable Iterable Iterator Generator Awaitable Coroutine Awaitable AsyncIterable AsyncIterator
(I'm excluding Hashable from the list, as the main reason that matters is in describing whether or not something is suitable for inclusion in a dict or set)
These differ from the rest of the collections.abc interfaces in that they're more closely associated with language level control flow syntax than they are with containers specifically:
Callable - function calls Iterable - for loops, comprehensions Iterator - for loops, comprehensions Generator - generators, generator expressions Awaitable - await expressions Coroutine - async def AsyncIterable - async for AsyncIterator - async for
Adding ContextManager and AsyncContextManager would give ABCs for the protocols related to "with" and "async with".
Since these all correspond to syntactic protocols, I now think it's reasonable to include them directly in the "abc" namespace, since that still gives us a clear guideline for which ABCs go there, and which belong somewhere else: if it has syntax associated with it, or it's part of the ABC machinery itself, then it can go directly in the "abc" namespace. Using the "abc" namespace also ensures there isn't any additional import overhead, since that gets imported by all ABC using code anyway.
Regards, Nick.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Dec 31, 2015 at 12:48 PM Brett Cannon <brett@python.org> wrote:
On Thu, Dec 31, 2015, 07:08 Alexander Walters <tritium-list@sdamon.com> wrote:
Would it be a good idea to mix 'concrete implementations of ABCs'* directly in the abc module where the tooling to create ABCs live, or to put it in a submodule? I feel it should be a submodule, but that isn't based on vast experience.
Locating collections ABCs in a submodule makes some sense, as there are 21 of them and the collections module is important for beginners to learn without getting distracted by ABCs. Contrast that with the direct inclusion of ABCs in most other modules and it suggests the creation of a submodule for collections may have been motivated for the same reason as this discussion -- it didn't feel right to have certain ABCs directly in the collections module. If the non-collection ABCs are being moved out of the collections module and into the ``abc`` module, there's less reason to separate them into a submodule. Beginners don't venture into the abc module expecting to understand everything. It's natural to find a bunch of ABCs in a module called ``abc``. And ABCs are included directly in many other modules instead of being relegated to a less discoverable submodule like ``typing.abc``, ``io.abc``, ``numbers.abc``, etc. as many of those are focused on ABCs in the first place. It's easy to notice a submodule when reading docs on the internet, but it's hard to figure out what the correct module is to import when hanging out at a basic REPL. Flat is better than nested and all that. * Yes, I know, these are not concrete implementations of types... I find
it a little confusing to describe.
It's a possibility. Feel free to comment on the issue if you want to discuss this further.
On 1 January 2016 at 08:18, Michael Selik <mike@selik.org> wrote:
On Thu, Dec 31, 2015 at 12:48 PM Brett Cannon <brett@python.org> wrote:
On Thu, Dec 31, 2015, 07:08 Alexander Walters <tritium-list@sdamon.com> wrote:
Would it be a good idea to mix 'concrete implementations of ABCs'* directly in the abc module where the tooling to create ABCs live, or to put it in a submodule? I feel it should be a submodule, but that isn't based on vast experience.
Locating collections ABCs in a submodule makes some sense, as there are 21 of them and the collections module is important for beginners to learn without getting distracted by ABCs. Contrast that with the direct inclusion of ABCs in most other modules and it suggests the creation of a submodule for collections may have been motivated for the same reason as this discussion -- it didn't feel right to have certain ABCs directly in the collections module.
No need to speculate, the original motive for the move is documented in the tracker: http://bugs.python.org/issue11085 (which can be found by looking at the commit history for the collections module: https://hg.python.org/cpython/log/3.5/Lib/collections/abc.py ) The problem was with folks getting confused between the abstract types like Sequence and Mapping and the usable classes like deque, ChainMap, OrderedDict, defaultdict, etc, rather than there being a lot of non-collections related content in the file. At the time, Callable was the only non-container related ABC in collections.abc - most of the others now being considered for relocation (Generator, Coroutine, Awaitable, AsyncIterable, AsyncIterator) were added as part of the PEP 492 implementation in 3.5, and *that* was mostly driven by Iterable and Iterator already being there so it was "logical" to also add Generator, AsyncIterable and AsyncIterator, with Coroutine and Awaitable coming along for the ride. That does raise the question of whether or not it's worth continuing to publish the PEP 492 ABCs from collections.abc - Guido formally accepted PEP 492 with provisional status [1], so we have scope to do the following: - add abc.(Generator, Coroutine, Awaitable, AsyncIterable, AsyncIterator) in 3.5.2 (keeping the aliases in collections.abc) - drop the collections.abc aliases for the PEP 492 ABCs in 3.6 - add abc.(Callable, Iterable, Iterator) in 3.6 (keeping the aliases in collections.abc indefinitely for Python 2 compatibility) [1] https://mail.python.org/pipermail/python-dev/2015-May/139844.html
If the non-collection ABCs are being moved out of the collections module and into the ``abc`` module, there's less reason to separate them into a submodule. Beginners don't venture into the abc module expecting to understand everything. It's natural to find a bunch of ABCs in a module called ``abc``. And ABCs are included directly in many other modules instead of being relegated to a less discoverable submodule like ``typing.abc``, ``io.abc``, ``numbers.abc``, etc. as many of those are focused on ABCs in the first place.
Right, the reason importlib.abc and collections.abc make sense is that when you import "importlib", you're probably interested in dynamic imports, and when you import "collections", you're probably interested in using one of the concrete container classes. The ABCs are only relevant if you're wanting to do some kind of type checking or define your own classes implementing the ABCs, so it makes sense to separate them at the module level, not just in the documentation. Other modules defining ABCs either don't need separation, or get their separation in other ways: abc: no separation needed, you're necessarily already thinking about ABCs when importing this typing: no separation needed, the only ABC is the one for defining generic types email: no separation needed, the only ABC is the one for defining email policies io: to use the io stack, you just call open() or use some other file/stream opening API numbers: to use the numeric tower, you use a builtin type, fractions.Fraction, decimal.Decimal, or some other numeric type Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Dec 31, 2015 11:48 AM, "Brett Cannon" <brett@python.org> wrote:
On Thu, Dec 31, 2015, 07:08 Alexander Walters <tritium-list@sdamon.com>
wrote:
Would it be a good idea to mix 'concrete implementations of ABCs'* directly in the abc module where the tooling to create ABCs live, or to put it in a submodule? I feel it should be a submodule, but that isn't based on vast experience.
* Yes, I know, these are not concrete implementations of types... I find it a little confusing to describe.
It's a possibility. Feel free to comment on the issue if you want to discuss this further.
-Brett
Are these interfaces? * | Docs: http://docs.zope.org/zope.interface/ * | Docs: http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/zca.html * | Docs: https://github.com/Pylons/pyramid/blob/master/pyramid/config/zca.py * | Docs: http://docs.pylonsproject.org/projects/pyramid/en/latest/api/interfaces.html * | Src: https://github.com/Pylons/pyramid/blob/master/pyramid/interfaces.py
On 12/28/2015 22:58, Nick Coghlan wrote:
On 29 December 2015 at 03:58, M.-A. Lemburg <mal@egenix.com> wrote:
On 28.12.2015 18:42, Brett Cannon wrote:
Speaking of using ABCs more, where should we put ABCs which have
do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__.
I opened http://bugs.python.org/issue25637 to discuss this, but I
nothing to figured a
wider discussion wouldn't hurt. Some suggest just putting the ABCs into the abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there.
Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc. I'd put them into the abc module (perhaps turning this into a package, if things get too crowded).
collections.abc could then do a "from abc import *" for b/w compatibility. With the benefit of hindsight, I think a broad namespace separation like that might have been a good way to go from a data model discoverability perspective, as it clearly separates the abstract descriptions of data and control flow modelling concepts from the concrete implementations of those concepts.
However, we should also keep in mind which standard library modules already publish ABCs, which is at least:
typing io numbers collections.abc selectors email
(That list was collected by grepping Python files for ABCMeta, so I may have missed some)
That suggests to me that this design decision has effectively already been made, and it's to include the ABCs in the relevant domain specific modules. The inclusion of the non-collections related ABCs in collections.abc are the anomaly, which can be addressed by moving them out to a more appropriate location (adjusting the documentation accordingly), and then importing them into collections.abc for backwards compatibility (taking care not to increase the startup import footprint too much in the process).
The current collections.abc interfaces which I think are most at issue here:
Callable Iterable Iterator Generator Awaitable Coroutine Awaitable AsyncIterable AsyncIterator
(I'm excluding Hashable from the list, as the main reason that matters is in describing whether or not something is suitable for inclusion in a dict or set)
These differ from the rest of the collections.abc interfaces in that they're more closely associated with language level control flow syntax than they are with containers specifically:
Callable - function calls Iterable - for loops, comprehensions Iterator - for loops, comprehensions Generator - generators, generator expressions Awaitable - await expressions Coroutine - async def AsyncIterable - async for AsyncIterator - async for
Adding ContextManager and AsyncContextManager would give ABCs for the protocols related to "with" and "async with".
Since these all correspond to syntactic protocols, I now think it's reasonable to include them directly in the "abc" namespace, since that still gives us a clear guideline for which ABCs go there, and which belong somewhere else: if it has syntax associated with it, or it's part of the ABC machinery itself, then it can go directly in the "abc" namespace. Using the "abc" namespace also ensures there isn't any additional import overhead, since that gets imported by all ABC using code anyway.
Regards, Nick.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On Dec 31, 2015 5:56 PM, "Wes Turner" <wes.turner@gmail.com> wrote:
On Dec 31, 2015 11:48 AM, "Brett Cannon" <brett@python.org> wrote:
On Thu, Dec 31, 2015, 07:08 Alexander Walters <tritium-list@sdamon.com>
wrote:
Would it be a good idea to mix 'concrete implementations of ABCs'* directly in the abc module where the tooling to create ABCs live, or to put it in a submodule? I feel it should be a submodule, but that isn't based on vast experience.
* Yes, I know, these are not concrete implementations of types... I
find
it a little confusing to describe.
It's a possibility. Feel free to comment on the issue if you want to discuss this further.
-Brett
Are these interfaces?
* | Docs: http://docs.zope.org/zope.interface/ * | Docs: http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/zca.html
* | Docs: https://github.com/Pylons/pyramid/blob/master/pyramid/config/zca.py * | Docs: http://docs.pylonsproject.org/projects/pyramid/en/latest/api/interfaces.html * | Src: https://github.com/Pylons/pyramid/blob/master/pyramid/interfaces.py
* pyramid.interfaces.IDict http://docs.pylonsproject.org/projects/pyramid/en/latest/api/interfaces.html... * pyramid.interfaces.IMultiDict http://docs.pylonsproject.org/projects/pyramid/en/latest/api/interfaces.html... [source]
On 12/28/2015 22:58, Nick Coghlan wrote:
On 29 December 2015 at 03:58, M.-A. Lemburg <mal@egenix.com> wrote:
On 28.12.2015 18:42, Brett Cannon wrote:
Speaking of using ABCs more, where should we put ABCs which have
do with collections? As of right now all ABCs seem to get shoved into collections.abc, but e.g. Awaitable and Coroutine are not types of collections. I personally want to add a context manager ABC with a default __exit__.
I opened http://bugs.python.org/issue25637 to discuss this, but I
wider discussion wouldn't hurt. Some suggest just putting the ABCs into the abc module. We could create an interfaces module (top-level or a submodule of ABC). The other option is to put the ABCs in subject-specific modules, so my context manager one would go into contextlib (either top-level or an abc submodule); don't know where the coroutine ones would go since it might be overloading asyncio if we out them there.
Anyway, the key point is collections.abc is starting to get non-collections stuff and if we are going to start pushing ABCs more we should decide how we want to organize them in general in the stdlib and instead of dumping them into collections.abc. I'd put them into the abc module (perhaps turning this into a package, if things get too crowded).
collections.abc could then do a "from abc import *" for b/w compatibility. With the benefit of hindsight, I think a broad namespace separation like that might have been a good way to go from a data model discoverability perspective, as it clearly separates the abstract descriptions of data and control flow modelling concepts from the concrete implementations of those concepts.
However, we should also keep in mind which standard library modules already publish ABCs, which is at least:
typing io numbers collections.abc selectors email
(That list was collected by grepping Python files for ABCMeta, so I may have missed some)
That suggests to me that this design decision has effectively already been made, and it's to include the ABCs in the relevant domain specific modules. The inclusion of the non-collections related ABCs in collections.abc are the anomaly, which can be addressed by moving
out to a more appropriate location (adjusting the documentation accordingly), and then importing them into collections.abc for backwards compatibility (taking care not to increase the startup import footprint too much in the process).
The current collections.abc interfaces which I think are most at issue here:
Callable Iterable Iterator Generator Awaitable Coroutine Awaitable AsyncIterable AsyncIterator
(I'm excluding Hashable from the list, as the main reason that matters is in describing whether or not something is suitable for inclusion in a dict or set)
These differ from the rest of the collections.abc interfaces in that they're more closely associated with language level control flow syntax than they are with containers specifically:
Callable - function calls Iterable - for loops, comprehensions Iterator - for loops, comprehensions Generator - generators, generator expressions Awaitable - await expressions Coroutine - async def AsyncIterable - async for AsyncIterator - async for
Adding ContextManager and AsyncContextManager would give ABCs for the protocols related to "with" and "async with".
Since these all correspond to syntactic protocols, I now think it's reasonable to include them directly in the "abc" namespace, since
nothing to figured a them that
still gives us a clear guideline for which ABCs go there, and which belong somewhere else: if it has syntax associated with it, or it's part of the ABC machinery itself, then it can go directly in the "abc" namespace. Using the "abc" namespace also ensures there isn't any additional import overhead, since that gets imported by all ABC using code anyway.
Regards, Nick.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
participants (6)
-
Alexander Walters -
Brett Cannon -
M.-A. Lemburg -
Michael Selik -
Nick Coghlan -
Wes Turner