On Mar 16, 2015, at 2:33 PM, Paul Moore
wrote: On 16 March 2015 at 17:14, Donald Stufft
wrote: The bulk of the effort of pushing the standards, pip, and PyPI through is done by a handful of people, and of those handful I believe that the largest share is done by myself. That's not to toot my own horn or any such nonsense but to simply state the fact that the available bandwidth of people able and willing to work on problems is low. However the things we bless here as official are things which need to be able to last for a decade or more, which means that they do need careful consideration before we bless them.
As a serious question - is there anything I (or indeed anyone else) can do to make progress on PEP 426? If I'm honest, right now I don't exactly see what tool changes are needed to change it from draft to accepted to actually implemented.
As far as I can see, acceptance consists largely of someone, somehow, confirming that there are no major loopholes in the spec. I think that mostly comes down to the fact that no-one has raised objections since the PEP was published, plus someone with experience of some of the more difficult distribution scenarios sanity-checking things. And then, getting it implemented in tools.
I guess the tool side consists of:
1. Making pip write a pydist.json file when installing from wheel. 2. Making setuptools write pydist.json when installing from sdist, and when creating a sdist. 3. Making wheel write pydist.json when writing a wheel.
(Also, when distlib writes a wheel, it should write pydist.json, but I'm considering distlib as "non-core" for the sake of this discussion).
There's also presumably work to add support for specifying some of the new metadata in setup.py, which I guess is setuptools work again.
Have I missed anything crucial? Paul
PS I'm ignoring the "standard metadata extensions" PEP where console wrappers, and post-install scripts figure. Those are probably bigger design issues.
Probably similar to what I did for PEP 440. Start branches implementing it and try to run as much stuff through it as possible to see how it fails. When implementing PEP 440 I was running every version number that existed on PyPI through it to test to make sure things would work. That’s not going to be possible with PEP 426, but ideally we should be able to get branches in the various tools that work on it, grab the latest versions of the top N packages (or the latest versions of everything) and compare the results. Another thing is determining if there's anything else we can/should split out from PEP 426 to narrow the scope. A quick skim to refresh myself doesn't show me anything that stands out, but some thought to this wouldn't hurt. It’d also probably not hurt to go through the setuptools and pip bug trackers to see if there are any relevant issues and see how they would be effected by the new standard. PEP 426 itself isn't much in the way of groundbreaking. It's basically taking the dynamic metadata and "compiling" it down into a static form which is JSON based. The "easy" metadata (name etc) is a no brainer and probably doesn't require much poking, trying out an example of an extension probably wouldn't be the worst thing either. Even if stanardizing script wrappers, for instance, is held off, getting a demo extension using it would validate the standard. Potentially more thought/guidance should be mentioned in how projects should straddle the line between legacy systems. If a new style metadata exists in a format in addition to an old syle should the new style be preferred? Should there be any sanity checks to make sure the two aren't completely different? As an aside, one thing I've had in the back of my mind, is looking at the possibility of defining the environment markers which specify versions as PEP 440 versions and enabling all the same comparisons with them as we get in the specifiers. I think the string based comparison we currently do is kinda janky and any installer already has the tooling to handle specifiers. Generally what I did with PEP 440, which I think worked well, is I had everything pretty much implemented prior to it get accepting, and we were then able to use the things I found out while implementing it to adjust the PEP before it was formally accepted. I just didn’t merge anything until that point. This was pretty valuable in findings things where the PEP was too vague for someone to make an indepdent implementation going by what was in the PEP, or if it specified something where implementing it turned out to be hard/problematic, etc. A major reason why I’m personally focusing on Warehouse first is that integrating with Warehouse will be easier than integrating with PyPI legacy. However that doesn’t have to block anyone else, that’s just myself not wanting to spend the time integrating a new major metadata version into the old legacy code base. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA