[Python-ideas] Python 3.9.9 - The 'I Have A Dream' Version

[Andrew Barnert]

On Apr 14, 2015, at 07:58, Simon Kennedy <sffjunkie at gmail.com> wrote:
> 
>> On Tuesday, 14 April 2015 14:01:30 UTC+1, Steven D'Aprano wrote:
>> 
>> Well, you've certainly been meticulous :-) 
>> 
>> [...] 
>> > * Removing the notion of 'virtual' environments. Everything's just an 
>> > environment. 
>> 
>> What's the difference?
> 
> Not a lot ;-) Merely a reduction in concepts by 1.
> 
> The question I asked myself when I thought about it is. Why 'virtual'? What's the conceptual difference between a standard Python install and a virtualenv (now that no-site-packages is the default.)? How does it help a Python user to distinguish between the two? Why do we have py.exe which finds a Python install but not a virtualenv?
> 
> So why not merge the Python install and the virtual environment into a single concept - Python Environments
> 
> Then we could have two tools
> 
> 1. 'py' which finds and starts environments. (plus py could automatically find scripts in the bin / scripts / tools directories without having to add them to the path)

How exactly does it do that? One of the nice things about virtual environments is that I can put them anywhere--in my home directory, on a network share, on an external drive, etc. I can make them relocatable and deploy them just by copying a directory. And so on. How could a tool possibly find all my environments?

More importantly, why would I ever want it to? The reason to have separate environments is so I can switch between them. A tool that treats them all the same as my base install is a tool that defeats the purpose of all of them.

And I certainly don't want anything installed in the bin/scripts/tools for any environment on my system to act like it's on my path even though it isn't.

> 2. 'pyenv' which manages environments (create, list, show info, activate etc.)
> 
> 
>> 
>> 
>> > * Splitting the standard library in twain. stdlib = 'baselib' + 'extlib'. 
>> >   - baselib = Enough to start up a Python environment / install pip. 
>> >   - extlib = The rest of the current stdlib. Changes more frequently than 
>> > the baselib. 
>> 
>> Hmmm. That would have consequences. Are you okay with abandoning large 
>> parts of the corporate, government and education sectors? 
>> 
>> The thing is, one of the reasons that Python is so successful is that it 
>> is *batteries included*. That's been a selling point for 20 years. This 
>> means that using only the standard library, you can be productive and 
>> get real work done. This is important for people using a standard 
>> operating environment, or in restricted networks with no access to the 
>> internet, or in any scenario where people have to get approval before 
>> they can use a particular software package. Not everyone is able to just 
>> download arbitrary packages from the Internet and use them on the spur 
>> of the moment. 
>> 
>> With Python, they only need to get *one* package approved: "Can I 
>> download Python and use it?" It might take six weeks to get approval, 
>> but once you do, you have not just the core language, but a good 150+ 
>> solid, useful, powerful libraries (and a few not so good ones). 
>> 
>> With your Python 3.9.9, they get the core language, and a mechanism for 
>> downloading from PyPI, which they are not allowed to do without 
>> approval, on pain of dismissal. Or not able to do, because the corporate 
>> network blocks it. And don't even think about sneaking it in via USB key 
>> from home... So each and every package that needs to be downloaded from 
>> PyPI could take six weeks to get approval before they can run "pip 
>> install os". 
>> 
>> The thing is, the broader Python ecosystem is not just hackers on Linux 
>> with control over their own machines. It includes people in schools 
>> using locked-down SOEs, corporate users with restricted network access 
>> or paranoid legal/IT departments, people in developing nations with 
>> unreliable internet access, and more.
> 
> The idea of the baselib + extlib is an attempt to address the desire to update parts of the stdlib faster than every Python release.
> 
> You would still get all the batteries included. When installing a new version of Python the extlib would be installed along with the baselib just not along side the baselib i.e. it would be in a separate on disk structure.
> 
> A new version of Python would be
> 
> * python.exe + baselib
> * extlib on disk structure somewhere else

So this is basically just PEP 413 in disguise. Have you read the discussions on that PEP, or at least the withdrawal reason?

You do add two additional things--moving most of the stdlib into an unspecified different location, and inventing a new tool (but just the name, not anything about what it actually does that couldn't already be done with pip) for doing the updates--but I'm not sure what either of those buys.

> The pippin tool which manages the extlib can pull in updates to the on disk repository in a batch as and when the maintainer wants.
> and the environment's maintainer can update the environment with the version they want.
> 
> Installation from the extlib disk structure to the current environment is 'just a copy' operation.
> 
> A difference then between a package on PyPi and in the extlib is only that the extlib is 'blessed' by the same people who have control of the Python releases.
> 
>  
>> 
>> > * A new tool called pippin to manage the extlib 
>> 
>> Does this actually exist, or have you picked it because the name is 
>> cool? (It is a cool name.)
> 
> Almost everything in the document doesn't exist yet and this in no exception. I think being a Tolkien fan may have something to do with why I picked it.
>  
>> 
>> (There is already a pippin on PyPI and github. I don't know if they are 
>> the same project. The one of github claims to be experimental. Remember 
>> that most experiments fail.) 
>> 
>> 
>> > * Formalising what a package index is 
>> 
>> What do you mean?
> 
> I can envision a future where there are multiple Python Package Indexes.
> 
> * The cheeseshop for 'uncontrolled' packages.
> * A company specific package index that only packages that have been signed by an administrator can be installed.
> * Plus others
> 
> Plus indexes in various stages of development
> 
> * PyPi - The current version
> * Warehouse - The next version
> * devpi - a bleeding edge development version
> 
> If there are to be tools which can interface with all these then there should be a formal document. Does this exist? I could not find one. All I've found various PEPS which propose updates to the cheeseshop.
> 
> This along with the pypi tool completes the PyPi Trinity.
>  
>> 
>> > * Addressing a Python project's lifecycle. 
>> 
>> Surely that will be different for every project?
> 
> Sort of, but every project must go through some of these stages.
> 
> Create, Design, Code, Build, Test, Create Distribution, Upload, Download, Install

Not every project needs all these stages. Not every project cycles through them in the same order. Many projects are developed by organizations that have their own process. Forcing everyone to do it the same way just means stopping many people from doing it at all.

> Consistent configuration between each of these steps would be desirable would it not?

Like what? Other than the small amount of information that's already in setup.py and the .cnf file, what other configuration do most projects need between any of these steps?

> Allowing the creation of tools to manage projects.

PyCharm, Eclipse PyDev, etc. already seem to be able to manage projects without anyone doing anything to allow it. What functionality are they missing that you want?

> If we don't look at them as a whole then we'll probably end up with inconsistencies.
> 
>  
>> 
>> > * Configuration handling 
>> 
>> What about it?
> 
> Because it's a thing that could be much more consistent across Python software.
> 
> What do we store in config files and where do we store them
> 
> * Sections
> * Item names
> * Item values (either singular or in a list)
> * Item types
> 
> All configuration file formats can handle these but there's no best practice.

No they can't. Plenty of configuration file formats can't handle sections; others handle unlimited levels of hierarchy. Some don't handle types, many handle small fixed sets of types (each different from the fixed set of types another format handles), only a few can handle arbitrary types. And so on.

> Why not formalise it?
> 
> Which directory should they go in? Again consistency is desirable. Surely we can come up with a 'blessed' set of locations?

Surely not. To the extent that you want that at all, it should be up to the OS, not Python. Consider what happens if Python says "config files on Unix must go in ~/.config", then Apple says "if you put config information anywhere but the defaults database inside your container, your app can't be sold on the App Store". So now nobody can write an app in Python and sell it in the App Store?

> Then document also attempts to address the layering of configurations across
> 
> * In Code
> * Site config
> * user config
> * Environment variables
> * Command line parameters

You almost certainly don't want that to work the same across every Python program. If I'm writing a tool in Python to do something MySQL-related, I want it to handle that stuff the same way as other MySQL-related tools, not the same way as other tools written in Python.
 
>> > * Removing the use of 'lib' suffixes from almost all packages. 
>> 
>> Why?
> 
> Because it's my vision of a future and I like consistency ;-) The question for me is why is it used in some places in the stdlib and not in others? What purpose does it serve? If it's a good idea shouldn't every package in the stdlib have a lib suffix? About the only place I can see it being useful is importlib because there's already an import statement and you don't want to 'import import' in your code
>  
>> 
>> > * Claiming any name in PyPi starting with 'py' 
>> 
>> That's going to be *really unpopular*. What is the purpose behind that?
> 
> I'm also sure it would be really unpopular but see the start of my previous response.
> 
> As it's only a vision of a possible future I'm under no illusion that it's the answer to all Python's 'problems' but it's an attempt to group all the disparate discussions into a single document.
> My answers may almost all be wrong / unimplementable but the discussions on the mailing list so far seem isolated and this is my attempt to make sense of them all.

Why do you think solving separate problems separately ("isolated") is a problem in the first place?

Why is discussing a jumble of dozens of barely-formed ideas, which are admittedly almost all wrong / unimplementable, as useful as discussing even one good idea on its own?

> 
> Regards
> Simon
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150414/f6188971/attachment-0001.html>