New workflow change: Welcome to blurb
One minor but ongoing problem we've had in CPython core development has been the mess of updating Misc/NEWS. Day-to-day developers may have a conflict if they lose a push race, which means a little editing. You'll have a similar, if slightly worse, problem when cherry-picking a fix between versions. Worst of all used to be the manual merges necessary after cutting a release--this was the bane of a CPython release manager's existence. (Though the new git-based workflow may have obviated the worst of this.)
The real problem is that we have one central file that everybody continually edits in a haphazard way. We aren't actually editing the same information, we aren't actually changing the same lines. But our revision control systems and diff algorithms don't understand the structure of Misc/NEWS and so they get confused. And for what? It's not like there's a tremendous benefit to having this central file everyone's fighting over.
We've been talking about addressing this for years. Fixing this was one of the goals of the new workflow. And finally, as of right now, the future is here. Ladies and gentlemen, I present: blurb.
https://github.com/python/core-workflow/tree/master/blurb
blurb is an interactive command-line tool that helps you write Misc/NEWS
entries. You simply run blurb from anywhere inside a CPython repo.
blurb runs an editor for you with a template open. You fill in these
three pieces of information:
- the bugs.python.org or "bpo" issue number,
- what "section" of Misc/NEWS this entry should go in (by uncommenting the correct line), and
- the text of the Misc/NEWS entry, in ReST format.
You save and exit and you're done. blurb even stages the Misc/NEWS entry in git for you!
Behind the scenes, blurb writes your information here:
Misc/NEWS.d/next/<section-name>/<filename>
The "<section-name>" is the name of the section in Misc/NEWS where your entry should go. <filename> contains the current date and time, the bpo number, and a nonce to prevent collisions.
These "next" files get merged together into a single aggregate .rst file
by the release manager when cutting a release (using "blurb release").
One nice feature of this approach: when you cherry-pick a change, its
Misc/NEWS entry in "next" gets cherry-picked along with it.
One important change: Misc/NEWS will no longer be checked in. It'll still be present in CPython tarballs; it will be generated by the release manager as part of cutting a release. But as a repository of information, it's been superseded by the various blurb data files. And by regenerating it from data files, we ensure that we'll never ever have a Misc/NEWS conflict ever again!
The plan is to leave Misc/NEWS in the CPython repo for maybe another week, to let the current crop of PRs get merged. But new work should switch to using blurb immediately.
You can install blurb from pip:
% pip3.6 install blurb
In fact--please do!
//arry/
One quick heads up – the NEWS file is included in the docs build (if not in the html docs, certainly in the CHM for Windows releases). You may have to do some extra work to keep that from breaking when you remove it. We might also include it as plain text in the installers, I forget right now.
Is blurb going to be embedded in the main repository? Not necessarily a problem if not, but I'd rather not have the build process depend on pip. Though I guess Sphinx is dependency already, so perhaps I should just integrate it better into the build…
Top-posted from my Windows phone
From: Larry Hastings Sent: Friday, June 23, 2017 20:26 To: Python Dev; python-committers Subject: [Python-Dev] New workflow change: Welcome to blurb
One minor but ongoing problem we've had in CPython core development has been the mess of updating Misc/NEWS. Day-to-day developers may have a conflict if they lose a push race, which means a little editing. You'll have a similar, if slightly worse, problem when cherry-picking a fix between versions. Worst of all used to be the manual merges necessary after cutting a release--this was the bane of a CPython release manager's existence. (Though the new git-based workflow may have obviated the worst of this.)
The real problem is that we have one central file that everybody continually edits in a haphazard way. We aren't actually editing the same information, we aren't actually changing the same lines. But our revision control systems and diff algorithms don't understand the structure of Misc/NEWS and so they get confused. And for what? It's not like there's a tremendous benefit to having this central file everyone's fighting over.
We've been talking about addressing this for years. Fixing this was one of the goals of the new workflow. And finally, as of right now, the future is here. Ladies and gentlemen, I present: blurb. https://github.com/python/core-workflow/tree/master/blurb
blurb is an interactive command-line tool that helps you write Misc/NEWS entries. You simply run blurb from anywhere inside a CPython repo. blurb runs an editor for you with a template open. You fill in these three pieces of information: • the bugs.python.org or "bpo" issue number, • what "section" of Misc/NEWS this entry should go in (by uncommenting the correct line), and • the text of the Misc/NEWS entry, in ReST format. You save and exit and you're done. blurb even stages the Misc/NEWS entry in git for you!
Behind the scenes, blurb writes your information here: Misc/NEWS.d/next/<section-name>/<filename> The "<section-name>" is the name of the section in Misc/NEWS where your entry should go. <filename> contains the current date and time, the bpo number, and a nonce to prevent collisions.
These "next" files get merged together into a single aggregate .rst file by the release manager when cutting a release (using "blurb release"). One nice feature of this approach: when you cherry-pick a change, its Misc/NEWS entry in "next" gets cherry-picked along with it.
One important change: Misc/NEWS will no longer be checked in. It'll still be present in CPython tarballs; it will be generated by the release manager as part of cutting a release. But as a repository of information, it's been superseded by the various blurb data files. And by regenerating it from data files, we ensure that we'll never ever have a Misc/NEWS conflict ever again!
The plan is to leave Misc/NEWS in the CPython repo for maybe another week, to let the current crop of PRs get merged. But new work should switch to using blurb immediately.
You can install blurb from pip: % pip3.6 install blurb In fact--please do!
/arry
On 24 June 2017 at 13:24, Larry Hastings larry@hastings.org wrote:
One minor but ongoing problem we've had in CPython core development has been the mess of updating Misc/NEWS. Day-to-day developers may have a conflict if they lose a push race, which means a little editing. You'll have a similar, if slightly worse, problem when cherry-picking a fix between versions. Worst of all used to be the manual merges necessary after cutting a release--this was the bane of a CPython release manager's existence. (Though the new git-based workflow may have obviated the worst of this.)
The real problem is that we have one central file that everybody continually edits in a haphazard way. We aren't actually editing the same information, we aren't actually changing the same lines. But our revision control systems and diff algorithms don't understand the structure of Misc/NEWS and so they get confused. And for what? It's not like there's a tremendous benefit to having this central file everyone's fighting over.
We've been talking about addressing this for years. Fixing this was one of the goals of the new workflow. And finally, as of right now, the future is here. Ladies and gentlemen, I present: blurb.
Thanks Larry, great to see this go live!
Behind the scenes, blurb writes your information here:
Misc/NEWS.d/next/<section-name>/<filename>
The "<section-name>" is the name of the section in Misc/NEWS where your entry should go. <filename> contains the current date and time, the bpo number, and a nonce to prevent collisions.
Folks are also free to handcraft these files if they really want to do so. The Developer Guide has the necessary details: https://docs.python.org/devguide/committing.html#what-s-new-and-news-entries
Cheers, Nick.
2017-06-24 6:24 GMT+03:00 Larry Hastings larry@hastings.org:
One minor but ongoing problem we've had in CPython core development has been the mess of updating Misc/NEWS. Day-to-day developers may have a conflict if they lose a push race, which means a little editing. You'll have a similar, if slightly worse, problem when cherry-picking a fix between versions. Worst of all used to be the manual merges necessary after cutting a release--this was the bane of a CPython release manager's existence. (Though the new git-based workflow may have obviated the worst of this.)
Thanks Larry! With the old hg-based workflow this was only slightly annoying, but the new git-based workflow turned this into a hell. If you have several PRs that updates the same Misc/NEWS section you needed to spent many time for just resolving conflicts one by one and waiting CI tests. And be lucky if other core developer trying to do the same withis PRs at the same time.
You can install blurb from pip:
% pip3.6 install blurb
In fact--please do!
I have installed it, but how to use it?
$ python3 -m pip install --user blurb Collecting blurb Using cached blurb-1.0-py3-none-any.whl Installing collected packages: blurb Successfully installed blurb-1.0 $ python3 -m blurb /usr/bin/python3: No module named blurb
On 06/23/2017 11:25 PM, Serhiy Storchaka wrote:
I have installed it, but how to use it?
$ python3 -m pip install --user blurb Collecting blurb Using cached blurb-1.0-py3-none-any.whl Installing collected packages: blurb Successfully installed blurb-1.0 $ python3 -m blurb /usr/bin/python3: No module named blurb
It's on your path. Just run
% blurb
from inside a CPython repo.
I'm amazed that your first thought was "python -m blurb".
//arry/
On 6/23/2017 11:24 PM, Larry Hastings wrote:
You can install blurb from pip:
% pip3.6 install blurb
This does not seem to work right. On Windows:
C:\Users\Terry>py -3 -m pip install blurb Collecting blurb Downloading blurb-1.0-py3-none-any.whl Installing collected packages: blurb Successfully installed blurb-1.0
Explorer shows that 3.6 site-packages has a 'blurb-1.0.dist-info' directory but neither blurb.py nor 'blurb/' is present. So the following are to be expected.
C:\Users\Terry>py -3 -m blurb C:\Programs\Python36\python.exe: No module named blurb
py -3
import blurb
... ModuleNotFoundError: No module named 'blurb'
Serhiy reported a similar problem on, I presume, some flavor of Linux.
tjr
On 06/24/2017 09:40 AM, Terry Reedy wrote:
On 6/23/2017 11:24 PM, Larry Hastings wrote:
You can install blurb from pip:
% pip3.6 install blurbThis does not seem to work right. On Windows:
C:\Users\Terry>py -3 -m pip install blurb Collecting blurb Downloading blurb-1.0-py3-none-any.whl Installing collected packages: blurb Successfully installed blurb-1.0
Explorer shows that 3.6 site-packages has a 'blurb-1.0.dist-info' directory but neither blurb.py nor 'blurb/' is present. So the following are to be expected.
C:\Users\Terry>py -3 -m blurb C:\Programs\Python36\python.exe: No module named blurb
py -3
import blurb
... ModuleNotFoundError: No module named 'blurb'
Serhiy reported a similar problem on, I presume, some flavor of Linux.
I replied to Serhiy; it's just "blurb", it's a command-line tool, it's not a package or a module. It should be a command on your path.
TBH I don't know if installation of a command-line tool like that works
on Windows. The tool itself was ported to Windows by Zach at the PyCon
core dev sprints last month, though that predates the PyPI work, and in
any case I could have broken the Windows support since then.
Unfortunately I'm no longer a qualified Windows developer, so if it
doesn't work on Windows I fear someone will have to send me a PR.
//arry/
On Sat, 24 Jun 2017 at 09:46 Larry Hastings larry@hastings.org wrote:
On 06/24/2017 09:40 AM, Terry Reedy wrote:
On 6/23/2017 11:24 PM, Larry Hastings wrote:
You can install blurb from pip:
% pip3.6 install blurbThis does not seem to work right. On Windows:
C:\Users\Terry>py -3 -m pip install blurb Collecting blurb Downloading blurb-1.0-py3-none-any.whl Installing collected packages: blurb Successfully installed blurb-1.0
Explorer shows that 3.6 site-packages has a 'blurb-1.0.dist-info' directory but neither blurb.py nor 'blurb/' is present. So the following are to be expected.
C:\Users\Terry>py -3 -m blurb C:\Programs\Python36\python.exe: No module named blurb
py -3
import blurb
... ModuleNotFoundError: No module named 'blurb'
Serhiy reported a similar problem on, I presume, some flavor of Linux.
I replied to Serhiy; it's just "blurb", it's a command-line tool, it's not a package or a module. It should be a command on your path.
TBH I don't know if installation of a command-line tool like that works on Windows. The tool itself was ported to Windows by Zach at the PyCon core dev sprints last month, though that predates the PyPI work, and in any case I could have broken the Windows support since then. Unfortunately I'm no longer a qualified Windows developer, so if it doesn't work on Windows I fear someone will have to send me a PR.
One of the great perks of python3 -m blurb is it avoids needing to care
about your PATH on any platform.
Anyway, the next release of blurb -- whether that's 1.0.0.post1 or a bigger
release -- will have a blurb.py as well as the entry point giving people
the blurb command. And people can also use pipsi if they want to install
blurb as more of a self-contained command-line app (at least on UNIX; don't
know about its support on Windows).
I just pushed blurb 1.0.0.post1 which re-packages everything using flit so
there's a blurb.py and an entry point for the blurb command. That should
meet everyone's needs for launching the tool.
On Sat, 24 Jun 2017 at 09:54 Brett Cannon brett@python.org wrote:
On Sat, 24 Jun 2017 at 09:46 Larry Hastings larry@hastings.org wrote:
On 06/24/2017 09:40 AM, Terry Reedy wrote:
On 6/23/2017 11:24 PM, Larry Hastings wrote:
You can install blurb from pip:
% pip3.6 install blurbThis does not seem to work right. On Windows:
C:\Users\Terry>py -3 -m pip install blurb Collecting blurb Downloading blurb-1.0-py3-none-any.whl Installing collected packages: blurb Successfully installed blurb-1.0
Explorer shows that 3.6 site-packages has a 'blurb-1.0.dist-info' directory but neither blurb.py nor 'blurb/' is present. So the following are to be expected.
C:\Users\Terry>py -3 -m blurb C:\Programs\Python36\python.exe: No module named blurb
py -3
import blurb
... ModuleNotFoundError: No module named 'blurb'
Serhiy reported a similar problem on, I presume, some flavor of Linux.
I replied to Serhiy; it's just "blurb", it's a command-line tool, it's not a package or a module. It should be a command on your path.
TBH I don't know if installation of a command-line tool like that works on Windows. The tool itself was ported to Windows by Zach at the PyCon core dev sprints last month, though that predates the PyPI work, and in any case I could have broken the Windows support since then. Unfortunately I'm no longer a qualified Windows developer, so if it doesn't work on Windows I fear someone will have to send me a PR.
One of the great perks of
python3 -m blurbis it avoids needing to care about your PATH on any platform.Anyway, the next release of blurb -- whether that's 1.0.0.post1 or a bigger release -- will have a blurb.py as well as the entry point giving people the
blurbcommand. And people can also use pipsi if they want to install blurb as more of a self-contained command-line app (at least on UNIX; don't know about its support on Windows).
On 6/24/2017 12:45 PM, Larry Hastings wrote:
On 06/24/2017 09:40 AM, Terry Reedy wrote:
On 6/23/2017 11:24 PM, Larry Hastings wrote:
You can install blurb from pip:
% pip3.6 install blurbThis does not seem to work right. On Windows:
C:\Users\Terry>py -3 -m pip install blurb Collecting blurb Downloading blurb-1.0-py3-none-any.whl Installing collected packages: blurb Successfully installed blurb-1.0
Explorer shows that 3.6 site-packages has a 'blurb-1.0.dist-info' directory but neither blurb.py nor 'blurb/' is present. So the following are to be expected.
C:\Users\Terry>py -3 -m blurb C:\Programs\Python36\python.exe: No module named blurb
py -3
import blurb
... ModuleNotFoundError: No module named 'blurb'
Serhiy reported a similar problem on, I presume, some flavor of Linux.
I replied to Serhiy; it's just "blurb", it's a command-line tool, it's not a package or a module. It should be a command on your path.
The reason I tried "<something> -m blurb" is because that is the standard and recommended way to run installed scripts on Windows. That is how I run pip and cherry_picker, for instance.
I found 'blurb' in <36dir>/Scripts/. The name and location are errors.
- On Windows, python files need the .py extension.
- That directory is not currently on the path on my machine. I believe
it once was, but installing 3.5.3 replaced it with the 3.5 /Scripts. On Windows, 3rd party installers must not presume that any /Scripts directory is on the path. By default, none are.
Solution: name the file blurb.py and put it in site-packages. This is standard and what is done by all other pip-installs that I have run. Put a copy in /Scripts if you want, but that is really optional and only sometimes effective.
TBH I don't know if installation of a command-line tool like that works on Windows. The tool itself was ported to Windows by Zach at the PyCon core dev sprints last month, though that predates the PyPI work, and in any case I could have broken the Windows support since then.
Unfortunately I'm no longer a qualified Windows developer, so if it doesn't work on Windows I fear someone will have to send me a PR.
I only know what the end result should be. Pip-installed Cherry_picker works on Windows, so copy from the spec files for that, or ask whoever wrote the pip-upload.
On 06/24/2017 10:30 AM, Terry Reedy wrote:
Solution: name the file blurb.py and put it in site-packages. This is standard and what is done by all other pip-installs that I have run. Put a copy in /Scripts if you want, but that is really optional and only sometimes effective.
Brett redid the installer with "flit" and pushed, and he says you should now be able to run blurb via "python3 -m blurb". Please update blurb (via pip3.6) and let us know if it now works for you on Windows.
Cheers,
//arry/
participants (6)
-
Brett Cannon -
Larry Hastings -
Nick Coghlan -
Serhiy Storchaka -
Steve Dower -
Terry Reedy