[Python-ideas] Where to put non-collection ABCs (was: Deprecating the old-style sequence protocol)

[Nick Coghlan]

On 1 January 2016 at 08:18, Michael Selik <mike at selik.org> wrote:
> On Thu, Dec 31, 2015 at 12:48 PM Brett Cannon <brett at python.org> wrote:
>>
>> On Thu, Dec 31, 2015, 07:08 Alexander Walters <tritium-list at sdamon.com>
>> wrote:
>>>
>>> Would it be a good idea to mix 'concrete implementations of ABCs'*
>>> directly in the abc module where the tooling to create ABCs live, or to
>>> put it in a submodule?  I feel it should be a submodule, but that isn't
>>> based on vast experience.
>
> Locating collections ABCs in a submodule makes some sense, as there are 21
> of them and the collections module is important for beginners to learn
> without getting distracted by ABCs. Contrast that with the direct inclusion
> of ABCs in most other modules and it suggests the creation of a submodule
> for collections may have been motivated for the same reason as this
> discussion -- it didn't feel right to have certain ABCs directly in the
> collections module.

No need to speculate, the original motive for the move is documented
in the tracker: http://bugs.python.org/issue11085 (which can be found
by looking at the commit history for the collections module:
https://hg.python.org/cpython/log/3.5/Lib/collections/abc.py )

The problem was with folks getting confused between the abstract types
like Sequence and Mapping and the usable classes like deque, ChainMap,
OrderedDict, defaultdict, etc, rather than there being a lot of
non-collections related content in the file.

At the time, Callable was the only non-container related ABC in
collections.abc - most of the others now being considered for
relocation (Generator, Coroutine, Awaitable, AsyncIterable,
AsyncIterator) were added as part of the PEP 492 implementation in
3.5, and *that* was mostly driven by Iterable and Iterator already
being there so it was "logical" to also add Generator, AsyncIterable
and AsyncIterator, with Coroutine and Awaitable coming along for the
ride.

That does raise the question of whether or not it's worth continuing
to publish the PEP 492 ABCs from collections.abc - Guido formally
accepted PEP 492 with provisional status [1], so we have scope to do
the following:

- add abc.(Generator, Coroutine, Awaitable, AsyncIterable,
AsyncIterator) in 3.5.2 (keeping the aliases in collections.abc)
- drop the collections.abc aliases for the PEP 492 ABCs in 3.6
- add abc.(Callable, Iterable, Iterator) in 3.6 (keeping the aliases
in collections.abc indefinitely for Python 2 compatibility)

[1] https://mail.python.org/pipermail/python-dev/2015-May/139844.html

> If the non-collection ABCs are being moved out of the collections module and
> into the ``abc`` module, there's less reason to separate them into a
> submodule. Beginners don't venture into the abc module expecting to
> understand everything. It's natural to find a bunch of ABCs in a module
> called ``abc``. And ABCs are included directly in many other modules instead
> of being relegated to a less discoverable submodule like ``typing.abc``,
> ``io.abc``, ``numbers.abc``, etc. as many of those are focused on ABCs in
> the first place.

Right, the reason importlib.abc and collections.abc make sense is that
when you import "importlib", you're probably interested in dynamic
imports, and when you import "collections", you're probably interested
in using one of the concrete container classes. The ABCs are only
relevant if you're wanting to do some kind of type checking or define
your own classes implementing the ABCs, so it makes sense to separate
them at the module level, not just in the documentation.

Other modules defining ABCs either don't need separation, or get their
separation in other ways:

abc: no separation needed, you're necessarily already thinking about
ABCs when importing this
typing: no separation needed, the only ABC is the one for defining generic types
email: no separation needed, the only ABC is the one for defining email policies
io: to use the io stack, you just call open() or use some other
file/stream opening API
numbers: to use the numeric tower, you use a builtin type,
fractions.Fraction, decimal.Decimal, or some other numeric type

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia