[Python-ideas] Where to put non-collection ABCs (was: Deprecating the old-style sequence protocol)

[Nick Coghlan]

On 29 December 2015 at 03:58, M.-A. Lemburg <mal at egenix.com> wrote:
> On 28.12.2015 18:42, Brett Cannon wrote:
>> Speaking of using ABCs more, where should we put ABCs which have nothing to
>> do with collections? As of right now all ABCs seem to get shoved into
>> collections.abc, but e.g. Awaitable and Coroutine are not types of
>> collections. I personally want to add a context manager ABC with a default
>> __exit__.
>>
>> I opened http://bugs.python.org/issue25637 to discuss this, but I figured a
>> wider discussion wouldn't hurt. Some suggest just putting the ABCs into the
>> abc module. We could create an interfaces module (top-level or a submodule
>> of ABC). The other option is to put the ABCs in subject-specific modules,
>> so my context manager one would go into contextlib (either top-level or an
>> abc submodule); don't know where the coroutine ones would go since it might
>> be overloading asyncio if we out them there.
>>
>> Anyway, the key point is collections.abc is starting to get non-collections
>> stuff and if we are going to start pushing ABCs more we should decide how
>> we want to organize them in general in the stdlib and instead of dumping
>> them into collections.abc.
>
> I'd put them into the abc module (perhaps turning this into a
> package, if things get too crowded).
>
> collections.abc could then do a "from abc import *" for b/w
> compatibility.

With the benefit of hindsight, I think a broad namespace separation
like that might have been a good way to go from a data model
discoverability perspective, as it clearly separates the abstract
descriptions of data and control flow modelling concepts from the
concrete implementations of those concepts.

However, we should also keep in mind which standard library modules
already publish ABCs, which is at least:

  typing
  io
  numbers
  collections.abc
  selectors
  email

(That list was collected by grepping Python files for ABCMeta, so I
may have missed some)

That suggests to me that this design decision has effectively already
been made, and it's to include the ABCs in the relevant domain
specific modules. The inclusion of the non-collections related ABCs in
collections.abc are the anomaly, which can be addressed by moving them
out to a more appropriate location (adjusting the documentation
accordingly), and then importing them into collections.abc for
backwards compatibility (taking care not to increase the startup
import footprint too much in the process).

The current collections.abc interfaces which I think are most at issue here:

    Callable
    Iterable
    Iterator
    Generator
    Awaitable
    Coroutine
    Awaitable
    AsyncIterable
    AsyncIterator

(I'm excluding Hashable from the list, as the main reason that matters
is in describing whether or not something is suitable for inclusion in
a dict or set)

These differ from the rest of the collections.abc interfaces in that
they're more closely associated with language level control flow
syntax than they are with containers specifically:

    Callable - function calls
    Iterable - for loops, comprehensions
    Iterator - for loops, comprehensions
    Generator - generators, generator expressions
    Awaitable - await expressions
    Coroutine - async def
    AsyncIterable - async for
    AsyncIterator  - async for

Adding ContextManager and AsyncContextManager would give ABCs for the
protocols related to "with" and "async with".

Since these all correspond to syntactic protocols, I now think it's
reasonable to include them directly in the "abc" namespace, since that
still gives us a clear guideline for which ABCs go there, and which
belong somewhere else: if it has syntax associated with it, or it's
part of the ABC machinery itself, then it can go directly in the "abc"
namespace. Using the "abc" namespace also ensures there isn't any
additional import overhead, since that gets imported by all ABC using
code anyway.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia