Question about subpackage/submodule API
The recent discussion about what to call the module in Pim Schellart's nice Lomb-Scargle contribution brought up a question for me that I have not seen discussed before. In many cases, a subpackage contains many modules, and the assorted functions and classes in these modules are made available in the (sub)package-level namespace by importing them from within __init__.py. For example, scipy/stats contains distributions.py, kde.py, morestats.py, mstats.py, rv.py, stats.py and vonmises.py. Except for mstats.py, the contents of all these are imported into the __init__.py namespace and included in __all__, so users can say, for example,
from scipy.stats import bayes_mvs
instead of
from scipy.stats.more_stats import bayes_mvs
Is this an *intentional* definition of an API? For example, is it "wrong" for a user to refer to the submodule 'more_stats' explicitly when importing? I think the answer is yes, because I don't think any promise is being made that the submodule won't be renamed or refactored as scipy development progresses. But I'd like to be sure that that is everyone else's understanding. If that is not the case, then changes that I have made in the past have violated the deprecation policy. For example, a while back I moved the functions in signal/filter_design.py that were related to FIR filters to their own module, fir_filter_design.py. By making appropriate changes to signal/__init__.py, all the function were still importable from scipy.signal. So, from my point of view, I didn't change the public API and no deprecation was necessary. But if scipy.signal.filter_design is part of the public API, then I should have made sure that something like
from scipy.signal.filter_design import firwin
still worked after the refactor. Note: I'm not saying that *all* submodules should be private and have their objects exposed only through __init__.py. I'm just looking for some clarification of existing policy. Warren
On Sat, Feb 12, 2011 at 1:53 PM, Warren Weckesser <warren.weckesser@enthought.com> wrote:
The recent discussion about what to call the module in Pim Schellart's nice Lomb-Scargle contribution brought up a question for me that I have not seen discussed before. In many cases, a subpackage contains many modules, and the assorted functions and classes in these modules are made available in the (sub)package-level namespace by importing them from within __init__.py. For example, scipy/stats contains distributions.py, kde.py, morestats.py, mstats.py, rv.py, stats.py and vonmises.py. Except for mstats.py, the contents of all these are imported into the __init__.py namespace and included in __all__, so users can say, for example,
from scipy.stats import bayes_mvs
instead of
from scipy.stats.more_stats import bayes_mvs
Is this an *intentional* definition of an API? For example, is it "wrong" for a user to refer to the submodule 'more_stats' explicitly when importing? I think the answer is yes, because I don't think any promise is being made that the submodule won't be renamed or refactored as scipy development progresses. But I'd like to be sure that that is everyone else's understanding.
If that is not the case, then changes that I have made in the past have violated the deprecation policy. For example, a while back I moved the functions in signal/filter_design.py that were related to FIR filters to their own module, fir_filter_design.py. By making appropriate changes to signal/__init__.py, all the function were still importable from scipy.signal. So, from my point of view, I didn't change the public API and no deprecation was necessary. But if scipy.signal.filter_design is part of the public API, then I should have made sure that something like
from scipy.signal.filter_design import firwin
still worked after the refactor.
Note: I'm not saying that *all* submodules should be private and have their objects exposed only through __init__.py. I'm just looking for some clarification of existing policy.
I haven't seen a discussion before either. I think the main reason to rename any modules is if there are not all objects or functions exposed in the namespace for the subpackage. I would never try to rename stats.distributions, because even though the distribution instances are exposed in stats, the classes itself are not. When we added mstats to the import I added mstats as import module fore the actual modules, as a sub namespace. Some modules have unexposed helper function that are sometimes useful for users to use. On the other hand, I don't think there is many import by users that is directly from stats.stats or stats.morestats or stats.mstats_extras. I think until scipy 1.0 there is still more streamlining in the scipy structure necessary, but I think we should move to a policy that also internal module reorganization is deprecated, even if this wasn't so in the past. If I remember correctly, there were other cases besides your changes. As a related aside: If we are ever allowed to make bigger changes, then I'd love to break up scipy.stats. I find the import time for scipy.stats (and the accompanying kitchen sink) pretty awful. Josef
Warren
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Sat, Feb 12, 2011 at 2:35 PM, <josef.pktd@gmail.com> wrote:
On Sat, Feb 12, 2011 at 1:53 PM, Warren Weckesser <warren.weckesser@enthought.com> wrote:
The recent discussion about what to call the module in Pim Schellart's nice Lomb-Scargle contribution brought up a question for me that I have not seen discussed before. In many cases, a subpackage contains many modules, and the assorted functions and classes in these modules are made available in the (sub)package-level namespace by importing them from within __init__.py. For example, scipy/stats contains distributions.py, kde.py, morestats.py, mstats.py, rv.py, stats.py and vonmises.py. Except for mstats.py, the contents of all these are imported into the __init__.py namespace and included in __all__, so users can say, for example,
from scipy.stats import bayes_mvs
instead of
from scipy.stats.more_stats import bayes_mvs
Is this an *intentional* definition of an API? For example, is it "wrong" for a user to refer to the submodule 'more_stats' explicitly when importing? I think the answer is yes, because I don't think any promise is being made that the submodule won't be renamed or refactored as scipy development progresses. But I'd like to be sure that that is everyone else's understanding.
If that is not the case, then changes that I have made in the past have violated the deprecation policy. For example, a while back I moved the functions in signal/filter_design.py that were related to FIR filters to their own module, fir_filter_design.py. By making appropriate changes to signal/__init__.py, all the function were still importable from scipy.signal. So, from my point of view, I didn't change the public API and no deprecation was necessary. But if scipy.signal.filter_design is part of the public API, then I should have made sure that something like
from scipy.signal.filter_design import firwin
still worked after the refactor.
Note: I'm not saying that *all* submodules should be private and have their objects exposed only through __init__.py. I'm just looking for some clarification of existing policy.
I haven't seen a discussion before either. I think the main reason to rename any modules is if there are not all objects or functions exposed in the namespace for the subpackage.
(proofreading after the send) I think the main reason *not* to rename a module is if it contains objects, functions and classes, that are not exposed in the namespace of the subpackage. Josef
I would never try to rename stats.distributions, because even though the distribution instances are exposed in stats, the classes itself are not. When we added mstats to the import I added mstats as import module fore the actual modules, as a sub namespace.
Some modules have unexposed helper function that are sometimes useful for users to use.
On the other hand, I don't think there is many import by users that is directly from stats.stats or stats.morestats or stats.mstats_extras.
I think until scipy 1.0 there is still more streamlining in the scipy structure necessary, but I think we should move to a policy that also internal module reorganization is deprecated, even if this wasn't so in the past. If I remember correctly, there were other cases besides your changes.
As a related aside: If we are ever allowed to make bigger changes, then I'd love to break up scipy.stats. I find the import time for scipy.stats (and the accompanying kitchen sink) pretty awful.
Josef
Warren
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
The policy in the past has been that the stable API is only one level down from the scipy namespace. So, developers should import the name from the top level namespace. As a subpackage grows, I could see justification for one more level in the stable API --- e.g. scipy.signal.spectral. But, I would be opposed to an API that is deeper than that. Travis -- (mobile phone of) Travis Oliphant Enthought, Inc. 1-512-536-1057 http://www.enthought.com On Feb 12, 2011, at 1:39 PM, josef.pktd@gmail.com wrote:
On Sat, Feb 12, 2011 at 2:35 PM, <josef.pktd@gmail.com> wrote:
On Sat, Feb 12, 2011 at 1:53 PM, Warren Weckesser <warren.weckesser@enthought.com> wrote:
The recent discussion about what to call the module in Pim Schellart's nice Lomb-Scargle contribution brought up a question for me that I have not seen discussed before. In many cases, a subpackage contains many modules, and the assorted functions and classes in these modules are made available in the (sub)package-level namespace by importing them from within __init__.py. For example, scipy/stats contains distributions.py, kde.py, morestats.py, mstats.py, rv.py, stats.py and vonmises.py. Except for mstats.py, the contents of all these are imported into the __init__.py namespace and included in __all__, so users can say, for example,
from scipy.stats import bayes_mvs
instead of
from scipy.stats.more_stats import bayes_mvs
Is this an *intentional* definition of an API? For example, is it "wrong" for a user to refer to the submodule 'more_stats' explicitly when importing? I think the answer is yes, because I don't think any promise is being made that the submodule won't be renamed or refactored as scipy development progresses. But I'd like to be sure that that is everyone else's understanding.
If that is not the case, then changes that I have made in the past have violated the deprecation policy. For example, a while back I moved the functions in signal/filter_design.py that were related to FIR filters to their own module, fir_filter_design.py. By making appropriate changes to signal/__init__.py, all the function were still importable from scipy.signal. So, from my point of view, I didn't change the public API and no deprecation was necessary. But if scipy.signal.filter_design is part of the public API, then I should have made sure that something like
from scipy.signal.filter_design import firwin
still worked after the refactor.
Note: I'm not saying that *all* submodules should be private and have their objects exposed only through __init__.py. I'm just looking for some clarification of existing policy.
I haven't seen a discussion before either. I think the main reason to rename any modules is if there are not all objects or functions exposed in the namespace for the subpackage.
(proofreading after the send) I think the main reason *not* to rename a module is if it contains objects, functions and classes, that are not exposed in the namespace of the subpackage.
Josef
I would never try to rename stats.distributions, because even though the distribution instances are exposed in stats, the classes itself are not. When we added mstats to the import I added mstats as import module fore the actual modules, as a sub namespace.
Some modules have unexposed helper function that are sometimes useful for users to use.
On the other hand, I don't think there is many import by users that is directly from stats.stats or stats.morestats or stats.mstats_extras.
I think until scipy 1.0 there is still more streamlining in the scipy structure necessary, but I think we should move to a policy that also internal module reorganization is deprecated, even if this wasn't so in the past. If I remember correctly, there were other cases besides your changes.
As a related aside: If we are ever allowed to make bigger changes, then I'd love to break up scipy.stats. I find the import time for scipy.stats (and the accompanying kitchen sink) pretty awful.
Josef
Warren
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Sat, 12 Feb 2011 14:12:44 -0600, Travis Oliphant wrote:
The policy in the past has been that the stable API is only one level down from the scipy namespace.
So, developers should import the name from the top level namespace.
As a subpackage grows, I could see justification for one more level in the stable API --- e.g. scipy.signal.spectral.
But, I would be opposed to an API that is deeper than that.
Agreed. One wild idea to make this clearer could be to prefix all internal sub- package names with the usual '_'. In the long run, it probably wouldn't be as bad as it initially sounds like. For instance, `numpy._core`, `scipy.special._orthogonal`, `scipy.linalg._decomp` etc. Pauli
On Sun, Feb 13, 2011 at 5:05 AM, Pauli Virtanen <pav@iki.fi> wrote:
On Sat, 12 Feb 2011 14:12:44 -0600, Travis Oliphant wrote:
The policy in the past has been that the stable API is only one level down from the scipy namespace.
So, developers should import the name from the top level namespace.
Is this written down somewhere? As far as I understand this is not standard practice in Python (one should use underscores). It's also at the moment not correct in parts of scipy, for example scipy.sparse.linalg.__all__ contains functions that are not available in sparse.__all__. Furthermore it would be quite natural to do something like: # 4 levels deep, from an actual bug report import scipy.sparse.linalg.eigen as eigen import scipy.signal.filter_design as filt
As a subpackage grows, I could see justification for one more level in
the stable API --- e.g. scipy.signal.spectral.
But, I would be opposed to an API that is deeper than that.
Agreed.
One wild idea to make this clearer could be to prefix all internal sub- package names with the usual '_'. In the long run, it probably wouldn't be as bad as it initially sounds like.
This is not a wild idea at all, I think it should be done. I considered all modules without '_' prefix public API.
Ralf
On Sat, Feb 12, 2011 at 7:28 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Sun, Feb 13, 2011 at 5:05 AM, Pauli Virtanen <pav@iki.fi> wrote:
On Sat, 12 Feb 2011 14:12:44 -0600, Travis Oliphant wrote:
The policy in the past has been that the stable API is only one level down from the scipy namespace.
So, developers should import the name from the top level namespace.
Is this written down somewhere? As far as I understand this is not standard practice in Python (one should use underscores). It's also at the moment not correct in parts of scipy, for example scipy.sparse.linalg.__all__ contains functions that are not available in sparse.__all__.
Furthermore it would be quite natural to do something like: # 4 levels deep, from an actual bug report import scipy.sparse.linalg.eigen as eigen import scipy.signal.filter_design as filt
As a subpackage grows, I could see justification for one more level in
the stable API --- e.g. scipy.signal.spectral.
But, I would be opposed to an API that is deeper than that.
Agreed.
One wild idea to make this clearer could be to prefix all internal sub- package names with the usual '_'. In the long run, it probably wouldn't be as bad as it initially sounds like.
This is not a wild idea at all, I think it should be done. I considered all modules without '_' prefix public API.
Agreed (despite what I said in my initial post). To actually do this, we'll need to check which packages have modules that should be private. These can be renamed in 0.10 to have an underscore, and new public versions created that contain a deprecation warning and that import everything from the private version. The deprecated public modules can be removed in 0.11. Some modules will require almost no changes. For example, scipy.cluster *only* exposes two modules, vq and hierarchy, so no changes are needed. (Well, there is also the module info.py that all packages have. That should become _info.py--there's no need for that to be public, is there?) Other packages will probably require some discussion about what modules should be public. Consider the above a proposed change for 0.10 and 0.11--what do you think? Warren
On Tue, Feb 15, 2011 at 7:53 AM, Warren Weckesser <warren.weckesser@enthought.com> wrote:
On Sat, Feb 12, 2011 at 7:28 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sun, Feb 13, 2011 at 5:05 AM, Pauli Virtanen <pav@iki.fi> wrote:
One wild idea to make this clearer could be to prefix all internal sub- package names with the usual '_'. In the long run, it probably wouldn't be as bad as it initially sounds like.
This is not a wild idea at all, I think it should be done. I considered all modules without '_' prefix public API.
Agreed (despite what I said in my initial post).
To actually do this, we'll need to check which packages have modules that should be private. These can be renamed in 0.10 to have an underscore, and new public versions created that contain a deprecation warning and that import everything from the private version. The deprecated public modules can be removed in 0.11.
Some modules will require almost no changes. For example, scipy.cluster *only* exposes two modules, vq and hierarchy, so no changes are needed. (Well, there is also the module info.py that all packages have. That should become _info.py--there's no need for that to be public, is there?)
Agreed, rename to _info.py
Other packages will probably require some discussion about what modules should be public.
Consider the above a proposed change for 0.10 and 0.11--what do you think?
Sounds good. Attached is a file that goes through scipy sub-packages and checks their __all__ for modules. Those are public by definition (but this doesn't give you the whole API). It's pretty messy, for example: signal ====== bsplines filter_design fir_filter_design integrate interpolate linalg ltisys np numpy optimize scipy signaltools sigtools special spline types warnings waveforms wavelets windows That should be cleaned up. Then there are also public modules that don't show up of course (for example odr.models). How about doing the following? : 1. Start a doc, perhaps on the wiki, with a full list of public modules. 2. Put that doc at the beginning of the reference guide, as well as the relevant part in the docstring for each sub-package. 3. Clean up existing __all__, and add __all__ to sub-packages that don't have them yet. 4. Rename private modules, with suitable deprecation warning. Cheers, Ralf
On Sun, Feb 20, 2011 at 3:33 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Tue, Feb 15, 2011 at 7:53 AM, Warren Weckesser <warren.weckesser@enthought.com> wrote:
On Sat, Feb 12, 2011 at 7:28 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sun, Feb 13, 2011 at 5:05 AM, Pauli Virtanen <pav@iki.fi> wrote:
One wild idea to make this clearer could be to prefix all internal sub- package names with the usual '_'. In the long run, it probably wouldn't be as bad as it initially sounds like.
This is not a wild idea at all, I think it should be done. I considered all modules without '_' prefix public API.
Agreed (despite what I said in my initial post).
To actually do this, we'll need to check which packages have modules that should be private. These can be renamed in 0.10 to have an underscore, and new public versions created that contain a deprecation warning and that import everything from the private version. The deprecated public modules can be removed in 0.11.
Some modules will require almost no changes. For example, scipy.cluster *only* exposes two modules, vq and hierarchy, so no changes are needed. (Well, there is also the module info.py that all packages have. That should become _info.py--there's no need for that to be public, is there?)
Agreed, rename to _info.py
This can't actually be done very easily, because the info.py name is hardcoded in PackageLoader in numpy._import_tools.py. So if desired, it first has to be done in numpy.
Other packages will probably require some discussion about what modules should be public.
Consider the above a proposed change for 0.10 and 0.11--what do you think?
Sounds good. Attached is a file that goes through scipy sub-packages and checks their __all__ for modules. Those are public by definition (but this doesn't give you the whole API). It's pretty messy, for example:
signal ====== bsplines filter_design fir_filter_design integrate interpolate linalg ltisys np numpy optimize scipy signaltools sigtools special spline types warnings waveforms wavelets windows
That should be cleaned up. Then there are also public modules that don't show up of course (for example odr.models).
How about doing the following? : 1. Start a doc, perhaps on the wiki, with a full list of public modules. 2. Put that doc at the beginning of the reference guide, as well as the relevant part in the docstring for each sub-package. 3. Clean up existing __all__, and add __all__ to sub-packages that don't have them yet. 4. Rename private modules, with suitable deprecation warning.
I've done the uncontroversial part (3): https://github.com/rgommers/scipy/tree/refactor-private-modules This cleans up the sub-package namespaces quite a bit, which will also make for example tab-completion in IPython easier to use. For example, sp.signal.<TAB> gives now 147 results instead of 262. There is one thing I wasn't sure about: should arccos/arccosh/arcsinh/... stay exposed in the scipy.special namespace, even though they are numpy functions? Here is a complete list of modules that I think are part (or should be part) of the public API. I added modules because they are documented as being public in docs, or contain useful functions/objects that are not exposed one level up, or because the sub-package namespace is very large and could benefit from a subdivision. cluster ======= vq hierarchy constants ========= fftpack ======= integrate ========= vode interpolate =========== dfitpack io == arff idl matlab mmio netcdf wavfile linalg ====== calc_lwork cblas clapack fblas flapack flinalg lapack special_matrices maxentropy ========== <deprecate this module> misc ==== doccer pilutil ndimage ======= filters fourier interpolation io measurements morphology odr === models odrpack optimize ======== signal ====== bsplines filter_design fir_filter_design ltisys spectral spline waveforms wavelets windows sparse ====== sparse.linalg ============= umfpack spatial ======= distance special ======= stats ===== distributions mstats <add more after refactoring, Josef wants to break up module. I agree it's too large and slow to import.> weave ===== <didn't look at weave yet> Cheers, Ralf
On Wed, Mar 9, 2011 at 6:17 AM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Tue, Feb 15, 2011 at 7:53 AM, Warren Weckesser <warren.weckesser@enthought.com> wrote:
On Sat, Feb 12, 2011 at 7:28 PM, Ralf Gommers <
ralf.gommers@googlemail.com>
wrote:
On Sun, Feb 13, 2011 at 5:05 AM, Pauli Virtanen <pav@iki.fi> wrote:
One wild idea to make this clearer could be to prefix all internal
sub-
package names with the usual '_'. In the long run, it probably wouldn't be as bad as it initially sounds like.
This is not a wild idea at all, I think it should be done. I considered all modules without '_' prefix public API.
Agreed (despite what I said in my initial post).
To actually do this, we'll need to check which packages have modules
On Sun, Feb 20, 2011 at 3:33 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote: that
should be private. These can be renamed in 0.10 to have an underscore, and new public versions created that contain a deprecation warning and that import everything from the private version. The deprecated public modules can be removed in 0.11.
Some modules will require almost no changes. For example, scipy.cluster *only* exposes two modules, vq and hierarchy, so no changes are needed. (Well, there is also the module info.py that all packages have. That should become _info.py--there's no need for that to be public, is there?)
Agreed, rename to _info.py
This can't actually be done very easily, because the info.py name is hardcoded in PackageLoader in numpy._import_tools.py. So if desired, it first has to be done in numpy.
Other packages will probably require some discussion about what modules should be public.
Consider the above a proposed change for 0.10 and 0.11--what do you think?
Sounds good. Attached is a file that goes through scipy sub-packages and checks their __all__ for modules. Those are public by definition (but this doesn't give you the whole API). It's pretty messy, for example:
signal ====== bsplines filter_design fir_filter_design integrate interpolate linalg ltisys np numpy optimize scipy signaltools sigtools special spline types warnings waveforms wavelets windows
That should be cleaned up. Then there are also public modules that don't show up of course (for example odr.models).
How about doing the following? : 1. Start a doc, perhaps on the wiki, with a full list of public modules. 2. Put that doc at the beginning of the reference guide, as well as the relevant part in the docstring for each sub-package. 3. Clean up existing __all__, and add __all__ to sub-packages that don't have them yet. 4. Rename private modules, with suitable deprecation warning.
I've done the uncontroversial part (3): https://github.com/rgommers/scipy/tree/refactor-private-modules This cleans up the sub-package namespaces quite a bit, which will also make for example tab-completion in IPython easier to use. For example, sp.signal.<TAB> gives now 147 results instead of 262.
After Warren reviewed it (thanks!), I've just pushed this branch. If anyone noticed any functions that look like they've gone missing then that's probably my fault. Easy to fix anyway and should get test coverage up.
There is one thing I wasn't sure about: should arccos/arccosh/arcsinh/... stay exposed in the scipy.special namespace, even though they are numpy functions?
I left these out of the scipy.special namespace, they're numpy functions after all. If anyone thinks this needs a deprecation let me know. The next step is to actually add underscores to non-public modules, I'll probably start on that after next week. Cheers, Ralf
Here is a complete list of modules that I think are part (or should be part) of the public API. I added modules because they are documented as being public in docs, or contain useful functions/objects that are not exposed one level up, or because the sub-package namespace is very large and could benefit from a subdivision.
cluster ======= vq hierarchy
constants =========
fftpack =======
integrate ========= vode
interpolate =========== dfitpack
io == arff idl matlab mmio netcdf wavfile
linalg ====== calc_lwork cblas clapack fblas flapack flinalg lapack special_matrices
maxentropy ========== <deprecate this module>
misc ==== doccer pilutil
ndimage ======= filters fourier interpolation io measurements morphology
odr === models odrpack
optimize ========
signal ====== bsplines filter_design fir_filter_design ltisys spectral spline waveforms wavelets windows
sparse ======
sparse.linalg ============= umfpack
spatial ======= distance
special =======
stats ===== distributions mstats <add more after refactoring, Josef wants to break up module. I agree it's too large and slow to import.>
weave ===== <didn't look at weave yet>
Cheers, Ralf
On Sat, Feb 12, 2011 at 19:28, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Sun, Feb 13, 2011 at 5:05 AM, Pauli Virtanen <pav@iki.fi> wrote:
On Sat, 12 Feb 2011 14:12:44 -0600, Travis Oliphant wrote:
The policy in the past has been that the stable API is only one level down from the scipy namespace.
So, developers should import the name from the top level namespace.
Is this written down somewhere? As far as I understand this is not standard practice in Python (one should use underscores).
Actually, it is de facto standard practice. The presence of underscores do mark something as private, but the absence of underscores do not mark something as public. Almost no one prepends underscores to module names regardless of whether they are considered "public" or "private". The public API of a package is determined by any number of conventions which may or may not be documented explicitly. For scipy, the convention is that public functions are exposed in the __init__.py. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Sun, 20 Feb 2011 11:40:27 -0600, Robert Kern wrote: [clip]
Actually, it is de facto standard practice. The presence of underscores do mark something as private, but the absence of underscores do not mark something as public. Almost no one prepends underscores to module names regardless of whether they are considered "public" or "private". The public API of a package is determined by any number of conventions which may or may not be documented explicitly. For scipy, the convention is that public functions are exposed in the __init__.py.
True, it is what is typically done. The reason why it's done like this is however usually more laziness than some well-thought-out design principle. This does not have to be so. I don't foresee prefixing an underscore to "private" modules to be a PITA in practice. Some people also add a "packagename.api" module to expose a well-defined API. It's true that this doesn't matter very much, but carefully painted bike sheds look nice. Pauli
On Mon, Feb 21, 2011 at 2:14 AM, Pauli Virtanen <pav@iki.fi> wrote:
On Sun, 20 Feb 2011 11:40:27 -0600, Robert Kern wrote: [clip]
Actually, it is de facto standard practice. The presence of underscores do mark something as private, but the absence of underscores do not mark something as public. Almost no one prepends underscores to module names regardless of whether they are considered "public" or "private". The public API of a package is determined by any number of conventions which may or may not be documented explicitly.
I tried to find a clear explanation somewhere in the Python docs but failed. So you're right. On the other hand Python itself seems to be consistent with underscored modules, and in the first Stackoverflow hit for "python API __init__" Alex Martelli says the same thing I did. I could be in worse company:)
For scipy, the convention is that public functions are exposed in the __init__.py.
Only one level down from the scipy namespace. Unless it's two. With a lot of stuff exposed that obviously isn't part of the API. It's pretty inconsistent.
True, it is what is typically done. The reason why it's done like this is however usually more laziness than some well-thought-out design principle.
This does not have to be so. I don't foresee prefixing an underscore to "private" modules to be a PITA in practice. Some people also add a "packagename.api" module to expose a well-defined API. It's true that this doesn't matter very much, but carefully painted bike sheds look nice.
Once you accidentally break users' code with a refactor like the one that started this thread, it may matter a little.... Cheers, Ralf
participants (6)
-
josef.pktd＠gmail.com -
Pauli Virtanen -
Ralf Gommers -
Robert Kern -
Travis Oliphant -
Warren Weckesser