[Python-Dev] Breaking undocumented API

[Michael Foord]

On 11/11/2010 05:41, Alexander Belopolsky wrote:
> On Wed, Nov 10, 2010 at 6:10 PM, Ron Adam<rrr at ronadam.com>  wrote:
> ..
>>> On Nov 10, 2010, at 5:47 AM, Michael Foord wrote:
>>>> So it is obvious that we don't have a clearly stated policy for what
>>>> defines the public API of standard library modules.
>>>>
>>>> How about making this explicit (either pep 8 or our developer docs):
>>> ..
>> The way I read Guido's email is that it is a situation dependent judgment
>> call for those cases that aren't clear.
>>
>> I think what Micheal is trying to say is for us to agree on some things so
>> we can go forward with a little more clarity.
> I don't understand why everyone seem to have accepted Michael's
> premise that "we don't have a clearly stated policy for what defines
> the public API of standard library modules."  We do have such a policy
> and it is well known (while the location in the reference manual may
> not be):

Ha. 14 paragraphs into the grammar reference on the import statement is 
perhaps not where developers would go to look for Python standard 
library development policy (and it *isn't* where they should go - 
standard library policy should be in pep 8 or our developer docs).

What you're saying is that the behaviour of "import *" *already* defines 
the public API at module level (but says nothing about class members or 
modules whose names begin with a leading underscore - those rules follow 
as a natural extension though).

By "clearly stated", I meant part of the python development 
documentation and / or standard library documentation. This is so that 
both users and developers are clear about the rules, and we have 
somewhere obvious to point people to. From this discussion it is clear 
that developers *don't* have a common understanding about what defines 
the public API of a standard library module. Suggestions as to what the 
rule is have included "only documented APIs are public" and "every 
member with a docstring is public"...

This largely comes from the heritage of the standard library which, as 
you point out, pre-dates the addition of __all__ / import * behaviour to 
the language. However many newer modules don't define __all__ either and 
several core developers have said they don't consider it a requirement 
that they do (as __all__ is a maintenance burden).

> """
> The public names defined by a module are determined by checking the
> module’s namespace for a variable named __all__;
> if defined, it must
> be a sequence of strings which are names defined or imported by that
> module. The names given in __all__ are all considered public and are
> required to exist. If __all__ is not defined, the set of public names
> includes all names found in the module’s namespace which do not begin
> with an underscore character ('_'). __all__ should contain the entire
> public API. It is intended to avoid accidentally exporting items that
> are not part of the API (such as library modules which were imported
> and used within the module).
> """  --<http://docs.python.org/reference/simple_stmts.html>
>
> The question that I had when I started this thread was not about a
> definition of "public API."  It was about a policy with respect to
> modules that precede the introduction of __all__ and the modern
> definition of public names.  (See r18692 "Two changes to
> from...import", and r23920 ' adding a definition of "public names"'.)
>

Well - restated your question is asking if adding a __all__ *changes* 
the public API of a standard library module. If it does then it is has 
stronger backwards compatibility implications than if it doesn't.

So given a standard library module that doesn't define __all__, what is 
considered the public API?

> Is it OK to add __all__ to such modules that does not include all
> names not starting with an underscore?  Is it OK to then remove names
> that clearly were not intended to be public?

Given the rules I suggested, which are basically the same as the one 
*you* are saying are already in place, if "import *" exports these names 
then you shouldn't change that behaviour without going through the 
deprecation process.

It would be clearer if these rules were stated either in pep 8 or our 
developer documentation of course.

All the best,

Michael

> Case in point: trace.rx_blank.  See also<http://bugs.python.org/issue10371>.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 

http://www.voidspace.org.uk/

READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.