I'd like to extend the proposal to Python 2.7 and later.
I don't object, but I also don't want to propose this, so I added it to the discussion. My (and perhaps other people's) concern is that 2.7 might well be the last release of the 2.x series. If so, adding this feature to it would make 2.7 an odd special case for users and providers of third party tools.
That's going to slow down Python package detection a lot - you'd replace an O(1) test with an O(n) scan.
I question that claim. In traditional Unix systems, the file system driver performs a linear search of the directory, so it's rather O(n)-in-kernel vs. O(n)-in-Python. Even for advanced file systems, you need at least O(log n) to determine whether a specific file is in a directory. For all practical purposes, the package directory will fit in a single disk block (containing a single .pkg file, and one or few subpackages), making listdir complete as fast as stat.
Wouldn't it be better to stick with a simpler approach and look for "__pkg__.py" files to detect namespace packages using that O(1) check ?
Again - this wouldn't be O(1). More importantly, it breaks system packages, which now again have to deal with the conflicting file names if they want to install all portions into a single location.
This would also avoid any issues you'd otherwise run into if you want to maintain this scheme in an importer that doesn't have access to a list of files in a package directory, but is well capable for the checking the existence of a file.
Do you have a specific mechanism in mind? Regards, Martin