PEP 518 and the pyproject.toml format
Hello,
First post, so short introduction: I'm mostly working on Django projects, currently for 3yourminD in Berlin. Working with python / Django for 6 going on 7 years. Also experienced in PHP, some Js and build systems like docker. We're currently migrating everything to docker and pip / requirements.txt isn't cutting it anymore and I've tasked myself with looking for or building an alternative tool.
This weekend I did quite a bit of research into the packaging tools and came accross PEP 518. What it doesn't address is that there now is a file resvered that declares very little. It does not declare and define sections to serve as a deployment file for a project and apparently it is unclear to tool builders what the one section is does define is supposed to be used for.
Case in point: poetry doesn't add itself to the build-system table when writing the py-project.toml file. Which, as I read it, it should. Ideally, every tool.${name} should be in the build system?
So, can we improve the file specification and introduce sections that allows a project (not a library) to specify how it should be built? Right now, this is all off-loaded to the tools that use the file and one cannot rely on anything. It also makes migration from one tool to another difficult and cooperation between tools hard. Or is this intentional: is this supposed to be tied to specific tools as build system are generally tightly coupled? And should we not change that?
It would help if the PEP had some explanation of the choices made to reserve a file that partially sets out to replace (all?) other files, but does not do anything to accomplish that. I realize that PEP-517 defines the API and leaves the implementation up to the tool, but that doesn't necessarily mean the py- project file should not name / reserve tables that the PEP 517 API should use.
I'm considering writing a tool that focuses solely on the deployment and dependency maintenance of a project and would like to not "vendor lock-in" my users, with even those dependency lists, because I have to use the tool namespace and can fill it every way I see fit and so will others.
So ideally, we would reserve dependency tables: [dependencies.$name]
Reserved names: standard, dev, test
Tools can add more names which they can then interpret themselves: [dependencies.stage] django-anonymizer = "^0.5"
And test tables and probably one or two more that come to mind. Is there any interest from the team to improve the file format and lock down such tables? I'd love to do the initial proposal if that helps move things forward.
On Mon, 10 Sep 2018 at 10:13, Melvyn Sopacua m.r.sopacua@gmail.com wrote:
First post, so short introduction:
Hi, and welcome.
I'm mostly working on Django projects, currently for 3yourminD in Berlin. Working with python / Django for 6 going on 7 years. Also experienced in PHP, some Js and build systems like docker. We're currently migrating everything to docker and pip / requirements.txt isn't cutting it anymore and I've tasked myself with looking for or building an alternative tool.
This weekend I did quite a bit of research into the packaging tools and came accross PEP 518. What it doesn't address is that there now is a file resvered that declares very little. It does not declare and define sections to serve as a deployment file for a project and apparently it is unclear to tool builders what the one section is does define is supposed to be used for.
It is true, PEP 518 doesn't define a lot by itself. But as it notes, "Tables not specified in this PEP are reserved for future use by other PEPs" - in other words, it's possible to add further details, but they should be backed by a formal specification, so that tools have a common understanding of what to do with the data.
Case in point: poetry doesn't add itself to the build-system table when writing the py-project.toml file. Which, as I read it, it should. Ideally, every tool.${name} should be in the build system?
Well, poetry isn't (as I understand it) a build system. It doesn't define *how* to build a wheel from project sources, rather it orchestrates the process of building an application from a set of projects (as I understand it from a brief read of the project page). That puts it in the same area as pipenv, I guess? That process (building *applications*) isn't something that's been really looked at yet in terms of standardisation, so yes, it'd still very much fair game for something to be discussed and agreed.
One reason this hasn't yet been looked at, as I understand things, is that there are a lot of very different scenarios to consider - web applications (which I guess are your area of interest), standalone command line utilities like Mercurial or the Leo editor, orchestration tools like Ansible, background OS service processes, etc. And it's unlikely that a "one size fits all" approach would work, so there's a whole question of how any standard would establish its scope and boundaries. One note of caution - the attempts I've seen in the past to define "build processes" tend to promote a certain amount of confusion by assuming all projects are similar, and then getting bogged down when people with very different requirements come along with different assumptions and mistaken expectations.
So, can we improve the file specification and introduce sections that allows a project (not a library) to specify how it should be built? Right now, this is all off-loaded to the tools that use the file and one cannot rely on anything. It also makes migration from one tool to another difficult and cooperation between tools hard. Or is this intentional: is this supposed to be tied to specific tools as build system are generally tightly coupled? And should we not change that?
You say "a project (not a library)". I presume by that you mean something like the application building process I described above? Yes, there's not been any standardisation effort in this area yet, so the situation is as you describe. It's not intentional as such, simply that no-one has really stepped up to propose anything yet. And maybe because there aren't that many formal tools in this area - a lot of what goes on appears to be simply workflow and local standards. But certainly it's something that might be worth exploring.
It would help if the PEP had some explanation of the choices made to reserve a file that partially sets out to replace (all?) other files, but does not do anything to accomplish that.
PEP 518 defines a means of "Specifying Minimum Build System Requirements for Python Projects". It defines a new, flexible, file format to hold that information, and defers further definition of the file contents to future PEPs, but it was never (to my knowledge) a goal of that PEP to look at "all other files" and fit them into the pyproject.toml format. It did reserve a namespace "tool.XXX" for projects that choose to consider themselves as "build tools" - and a number of projects (including poetry, I believe) have made use of that. (And as you can see, this opens up some confusion over what constitutes a "build tool", in contrast to my fairly strict definition above).
PEP 517 extended the pyproject.toml format to add data for projects to define which build system they want to be used when they are built.
There's nothing yet for "building a project (not a library)". You could start with getting agreement on some terminology explaining what "a project (not a library)" is, and should be called :-) Would it cover pip? What about Mercurial? Ansible? A Django site? A blogging application backed by Django (or Flask, or whatever)? Maybe we need multiple terms...
I realize that PEP-517 defines the API and leaves the implementation up to the tool, but that doesn't necessarily mean the py- project file should not name / reserve tables that the PEP 517 API should use.
It does - tools.flit and tools.setuptools, and similar, for other build tools.
Yes, there are places where further standardisation would be good (for example, config_options has no defined semantics, which means tools like pip can do little beyond pass unmodified key/value pairs to the backend), but I don't think that's what you're talking about here. I don't think PEP 517 is relevant to your proposal, though (unless you expect a tool like poetry to add a PEP 517 backend?)
I'm considering writing a tool that focuses solely on the deployment and dependency maintenance of a project and would like to not "vendor lock-in" my users, with even those dependency lists, because I have to use the tool namespace and can fill it every way I see fit and so will others.
So ideally, we would reserve dependency tables: [dependencies.$name]
Reserved names: standard, dev, test
Tools can add more names which they can then interpret themselves: [dependencies.stage] django-anonymizer = "^0.5"
And test tables and probably one or two more that come to mind. Is there any interest from the team to improve the file format and lock down such tables? I'd love to do the initial proposal if that helps move things forward.
For those tables to *not* lock people into your tool, you'd need other tools to agree to use the same names (I assume there *are* other tools doing the same as you propose, otherwise there's no potential for "lock in" being an issue at this stage ;-)), and for that you'd need a standard. That's why PEP 518 has such keys "reserved for future PEPs". It's not to block suggestions like yours, but rather to enable them (in a way that everyone gets to buy into, rather than the most popular tool dictating implementation defined standards).
Paul
One minor clarification—Poetry os both an application and a library packaging tool. It however relies on the old, non-isolated Setuptools approach to do the building part under the hood, so (from my own understanding)
Poetry ≒ Pipenv + Flit - PEP 517 - PEP 518
Whether this is a good approach is another topic (well, I guess my position is implied as a Pipenv maintainer), and Paul’s explanation is still complete valid.
On 10/9, 2018, at 18:47, Paul Moore p.f.moore@gmail.com wrote:
On Mon, 10 Sep 2018 at 10:13, Melvyn Sopacua m.r.sopacua@gmail.com wrote:
First post, so short introduction:
Hi, and welcome.
I'm mostly working on Django projects, currently for 3yourminD in Berlin. Working with python / Django for 6 going on 7 years. Also experienced in PHP, some Js and build systems like docker. We're currently migrating everything to docker and pip / requirements.txt isn't cutting it anymore and I've tasked myself with looking for or building an alternative tool.
This weekend I did quite a bit of research into the packaging tools and came accross PEP 518. What it doesn't address is that there now is a file resvered that declares very little. It does not declare and define sections to serve as a deployment file for a project and apparently it is unclear to tool builders what the one section is does define is supposed to be used for.
It is true, PEP 518 doesn't define a lot by itself. But as it notes, "Tables not specified in this PEP are reserved for future use by other PEPs" - in other words, it's possible to add further details, but they should be backed by a formal specification, so that tools have a common understanding of what to do with the data.
Case in point: poetry doesn't add itself to the build-system table when writing the py-project.toml file. Which, as I read it, it should. Ideally, every tool.${name} should be in the build system?
Well, poetry isn't (as I understand it) a build system. It doesn't define *how* to build a wheel from project sources, rather it orchestrates the process of building an application from a set of projects (as I understand it from a brief read of the project page). That puts it in the same area as pipenv, I guess? That process (building *applications*) isn't something that's been really looked at yet in terms of standardisation, so yes, it'd still very much fair game for something to be discussed and agreed.
One reason this hasn't yet been looked at, as I understand things, is that there are a lot of very different scenarios to consider - web applications (which I guess are your area of interest), standalone command line utilities like Mercurial or the Leo editor, orchestration tools like Ansible, background OS service processes, etc. And it's unlikely that a "one size fits all" approach would work, so there's a whole question of how any standard would establish its scope and boundaries. One note of caution - the attempts I've seen in the past to define "build processes" tend to promote a certain amount of confusion by assuming all projects are similar, and then getting bogged down when people with very different requirements come along with different assumptions and mistaken expectations.
So, can we improve the file specification and introduce sections that allows a project (not a library) to specify how it should be built? Right now, this is all off-loaded to the tools that use the file and one cannot rely on anything. It also makes migration from one tool to another difficult and cooperation between tools hard. Or is this intentional: is this supposed to be tied to specific tools as build system are generally tightly coupled? And should we not change that?
You say "a project (not a library)". I presume by that you mean something like the application building process I described above? Yes, there's not been any standardisation effort in this area yet, so the situation is as you describe. It's not intentional as such, simply that no-one has really stepped up to propose anything yet. And maybe because there aren't that many formal tools in this area - a lot of what goes on appears to be simply workflow and local standards. But certainly it's something that might be worth exploring.
It would help if the PEP had some explanation of the choices made to reserve a file that partially sets out to replace (all?) other files, but does not do anything to accomplish that.
PEP 518 defines a means of "Specifying Minimum Build System Requirements for Python Projects". It defines a new, flexible, file format to hold that information, and defers further definition of the file contents to future PEPs, but it was never (to my knowledge) a goal of that PEP to look at "all other files" and fit them into the pyproject.toml format. It did reserve a namespace "tool.XXX" for projects that choose to consider themselves as "build tools" - and a number of projects (including poetry, I believe) have made use of that. (And as you can see, this opens up some confusion over what constitutes a "build tool", in contrast to my fairly strict definition above).
PEP 517 extended the pyproject.toml format to add data for projects to define which build system they want to be used when they are built.
There's nothing yet for "building a project (not a library)". You could start with getting agreement on some terminology explaining what "a project (not a library)" is, and should be called :-) Would it cover pip? What about Mercurial? Ansible? A Django site? A blogging application backed by Django (or Flask, or whatever)? Maybe we need multiple terms...
I realize that PEP-517 defines the API and leaves the implementation up to the tool, but that doesn't necessarily mean the py- project file should not name / reserve tables that the PEP 517 API should use.
It does - tools.flit and tools.setuptools, and similar, for other build tools.
Yes, there are places where further standardisation would be good (for example, config_options has no defined semantics, which means tools like pip can do little beyond pass unmodified key/value pairs to the backend), but I don't think that's what you're talking about here. I don't think PEP 517 is relevant to your proposal, though (unless you expect a tool like poetry to add a PEP 517 backend?)
I'm considering writing a tool that focuses solely on the deployment and dependency maintenance of a project and would like to not "vendor lock-in" my users, with even those dependency lists, because I have to use the tool namespace and can fill it every way I see fit and so will others.
So ideally, we would reserve dependency tables: [dependencies.$name]
Reserved names: standard, dev, test
Tools can add more names which they can then interpret themselves: [dependencies.stage] django-anonymizer = "^0.5"
And test tables and probably one or two more that come to mind. Is there any interest from the team to improve the file format and lock down such tables? I'd love to do the initial proposal if that helps move things forward.
For those tables to *not* lock people into your tool, you'd need other tools to agree to use the same names (I assume there *are* other tools doing the same as you propose, otherwise there's no potential for "lock in" being an issue at this stage ;-)), and for that you'd need a standard. That's why PEP 518 has such keys "reserved for future PEPs". It's not to block suggestions like yours, but rather to enable them (in a way that everyone gets to buy into, rather than the most popular tool dictating implementation defined standards).
Paul
Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/E...
On maandag 10 september 2018 12:47:16 CEST Paul Moore wrote:
On Mon, 10 Sep 2018 at 10:13, Melvyn Sopacua m.r.sopacua@gmail.com wrote:
First post, so short introduction:
Hi, and welcome.
I'm mostly working on Django projects, currently for 3yourminD in Berlin. Working with python / Django for 6 going on 7 years. Also experienced in PHP, some Js and build systems like docker. We're currently migrating everything to docker and pip / requirements.txt isn't cutting it anymore and I've tasked myself with looking for or building an alternative tool.
This weekend I did quite a bit of research into the packaging tools and came accross PEP 518. What it doesn't address is that there now is a file resvered that declares very little. It does not declare and define sections to serve as a deployment file for a project and apparently it is unclear to tool builders what the one section is does define is supposed to be used for.
It is true, PEP 518 doesn't define a lot by itself. But as it notes, "Tables not specified in this PEP are reserved for future use by other PEPs" - in other words, it's possible to add further details, but they should be backed by a formal specification, so that tools have a common understanding of what to do with the data.
Great. Time to read "how to write a pep" and get started.
Case in point: poetry doesn't add itself to the build-system table when writing the py-project.toml file. Which, as I read it, it should. Ideally, every tool.${name} should be in the build system?
Well, poetry isn't (as I understand it) a build system. It doesn't define *how* to build a wheel from project sources, rather it orchestrates the process of building an application from a set of projects (as I understand it from a brief read of the project page). That puts it in the same area as pipenv, I guess? That process (building *applications*) isn't something that's been really looked at yet in terms of standardisation, so yes, it'd still very much fair game for something to be discussed and agreed.
One reason this hasn't yet been looked at, as I understand things, is that there are a lot of very different scenarios to consider - web applications (which I guess are your area of interest), standalone command line utilities like Mercurial or the Leo editor, orchestration tools like Ansible, background OS service processes, etc. And it's unlikely that a "one size fits all" approach would work, so there's a whole question of how any standard would establish its scope and boundaries. One note of caution - the attempts I've seen in the past to define "build processes" tend to promote a certain amount of confusion by assuming all projects are similar, and then getting bogged down when people with very different requirements come along with different assumptions and mistaken expectations.
..
You say "a project (not a library)". I presume by that you mean something like the application building process I described above?
Ok, so there's the first confusion. My use of project is fostered first and foremost by django's "startproject", but also by the use of project in bitbucket, github and gitlab: a collection of libraries working towards a common goal, that is usually unique in composition.
That + build-system put me on the path to believe that it's similar to a Pipfile. And maybe there is a good need to introduce more then one file, with shared traits or even building upon each other. So for now, I'll use your definition of project further on.
Yes, there's not been any standardisation effort in this area yet, so the situation is as you describe. It's not intentional as such, simply that no-one has really stepped up to propose anything yet. And maybe because there aren't that many formal tools in this area - a lot of what goes on appears to be simply workflow and local standards. But certainly it's something that might be worth exploring.
It would help if the PEP had some explanation of the choices made to reserve a file that partially sets out to replace (all?) other files, but does not do anything to accomplish that.
PEP 518 defines a means of "Specifying Minimum Build System Requirements for Python Projects". It defines a new, flexible, file format to hold that information, and defers further definition of the file contents to future PEPs, but it was never (to my knowledge) a goal of that PEP to look at "all other files" and fit them into the pyproject.toml format.
There is a section "sticking with setup.cfg", that at least implies setup.cfg can be replaced, but everything I can put in there, has no reserved spot in pyproject.toml.
It did reserve a namespace "tool.XXX" for projects that choose to consider themselves as "build tools" - and a number of projects (including poetry, I believe) have made use of that. (And as you can see, this opens up some confusion over what constitutes a "build tool", in contrast to my fairly strict definition above).
Indeed it does. Especially since pip is less used to build the packages of a given project and more to install it on to the system - at least in my experience. Even in tutorials you'll see "python setup.py build" advocated.
PEP 517 extended the pyproject.toml format to add data for projects to define which build system they want to be used when they are built.
There's nothing yet for "building a project (not a library)". You could start with getting agreement on some terminology explaining what "a project (not a library)" is, and should be called :-) Would it cover pip? What about Mercurial? Ansible? A Django site? A blogging application backed by Django (or Flask, or whatever)? Maybe we need multiple terms...
Yes, that would sure help any standardization going forward here.
I realize that PEP-517 defines the API and leaves the implementation up to the tool, but that doesn't necessarily mean the py- project file should not name / reserve tables that the PEP 517 API should use.
It does - tools.flit and tools.setuptools, and similar, for other build tools.
Yes, there are places where further standardisation would be good (for example, config_options has no defined semantics, which means tools like pip can do little beyond pass unmodified key/value pairs to the backend), but I don't think that's what you're talking about here. I don't think PEP 517 is relevant to your proposal, though (unless you expect a tool like poetry to add a PEP 517 backend?)
There's a lot of similarity, but in the end I don't think a web application with all it's sensitive information in configuration files, would beneift from being packaged as a wheel. So I think the first distinction one should make is between "installable" and "deployable": installed on a system to be ready to use, or deployed to continously running systems, constantly in use and upgraded with minimal downtime. Maybe, deployable is out of scope for python packaging even.
That said, even if it's out of scope, it would help to have a standardization of what a project needs and provides (to resolve dependencies), what version it has (to resolve updating needs) and how it can be tested.
Further more, I think that ironically, Pypi.org would suffer the most from this tool.* namespace, since it'll need to implement all the "metadata" sections, "version" sections and custom-stuff sections from all tools to provide all the useful information it does now.
So while it may not be relevant for my specific usecase, I still believe that PEP517's input data should be standardized inside py-project without the tool namespace.
I'm considering writing a tool that focuses solely on the deployment and dependency maintenance of a project and would like to not "vendor lock-in" my users, with even those dependency lists, because I have to use the tool namespace and can fill it every way I see fit and so will others.
So ideally, we would reserve dependency tables: [dependencies.$name]
Reserved names: standard, dev, test
Tools can add more names which they can then interpret themselves: [dependencies.stage] django-anonymizer = "^0.5"
And test tables and probably one or two more that come to mind. Is there any interest from the team to improve the file format and lock down such tables? I'd love to do the initial proposal if that helps move things forward.
For those tables to *not* lock people into your tool, you'd need other tools to agree to use the same names
No, I'm advocating that those sections be formalized (through new PEPs), so that tools SHALL use those sections for those bits of the PEP 517 api and we don't need 27 (random) different parses to obtain a dependency list for a given package.
On Mon, 10 Sep 2018 at 13:12, Melvyn Sopacua m.r.sopacua@gmail.com wrote:
You say "a project (not a library)". I presume by that you mean something like the application building process I described above?
Ok, so there's the first confusion. My use of project is fostered first and foremost by django's "startproject", but also by the use of project in bitbucket, github and gitlab: a collection of libraries working towards a common goal, that is usually unique in composition.
OK, cool. I have little or no knowledge of (or interest in) web applications, so that is indeed a point of confusion. If you're only interested in web applications, that's fine, but I'm not the person to comment. If you;re interested in more general applications, then I could well have more to say, but it's really important to realise that (at least as far as I know, and people who know better than me have told me) general application deployment is very different from web application deployment.
That + build-system put me on the path to believe that it's similar to a Pipfile. And maybe there is a good need to introduce more then one file, with shared traits or even building upon each other. So for now, I'll use your definition of project further on.
I didn't intend to imply any similarity to pipfile - in my experience, pipenv/pipfile are not a good model for the sorts of application I develop.
It did reserve a namespace "tool.XXX" for projects that choose to consider themselves as "build tools" - and a number of projects (including poetry, I believe) have made use of that. (And as you can see, this opens up some confusion over what constitutes a "build tool", in contrast to my fairly strict definition above).
Indeed it does. Especially since pip is less used to build the packages of a given project and more to install it on to the system - at least in my experience. Even in tutorials you'll see "python setup.py build" advocated.
PEP 517 is intended to allow pip (and tools like it) to build wheels from source. In reality, the intention is that wherever possible, projects should supply wheels, so PEP 517 is aimed at what we'd like to be a niche use case. But in practice, not all projects provide wheels, and not all platforms have wheels commonly built for them.
Tools *other* than pip may well benefit from PEP 517, to give a standard way to say "build a wheel from this source tree", but at the time of writing PEP 517, no such projects existed as a reference, so it's possible PEP 517 may need extension should such use cases arise. But the focus of PEP 517 remains "build a wheel (or sdist, which can be used to build a wheel) from a source tree". So, for example, dependency resolution is out of scope of PEP 517.
PEP 517 extended the pyproject.toml format to add data for projects to define which build system they want to be used when they are built.
There's nothing yet for "building a project (not a library)". You could start with getting agreement on some terminology explaining what "a project (not a library)" is, and should be called :-) Would it cover pip? What about Mercurial? Ansible? A Django site? A blogging application backed by Django (or Flask, or whatever)? Maybe we need multiple terms...
Yes, that would sure help any standardization going forward here.
Great. That should keep you busy for a few months to start with :-) Seriously, we've touched on this in the past, and backed away from it because it's so hard to get a common agreement - I can't give specific references, and you may find it hard to locate anything in the archives as it's never really been a topic in its own right, more of a side issue, but don't assume it's straightforward.
I realize that PEP-517 defines the API and leaves the implementation up to the tool, but that doesn't necessarily mean the py- project file should not name / reserve tables that the PEP 517 API should use.
It does - tools.flit and tools.setuptools, and similar, for other build tools.
Yes, there are places where further standardisation would be good (for example, config_options has no defined semantics, which means tools like pip can do little beyond pass unmodified key/value pairs to the backend), but I don't think that's what you're talking about here. I don't think PEP 517 is relevant to your proposal, though (unless you expect a tool like poetry to add a PEP 517 backend?)
There's a lot of similarity, but in the end I don't think a web application with all it's sensitive information in configuration files, would beneift from being packaged as a wheel.
Oh, dear me no. I agree entirely. There;'s no way the wheel format is right for that sort of thing.
So I think the first distinction one should make is between "installable" and "deployable": installed on a system to be ready to use, or deployed to continously running systems, constantly in use and upgraded with minimal downtime. Maybe, deployable is out of scope for python packaging even.
I don't really understand what you mean by "deployable", which may give you an idea of how different our viewpoints are. Is running get-pip.py to install pip on your system "deploying"? I would consider it to be. What about "pip install tox" to download and install tox and all its dependencies? Again, in my view yes.
So I don't really know what you're meaning by the distinction between "installable" and "deployable". But taking your "constantly in use and upgraded with minimal downtime", I suspect that as you say that's out of scope for Python packaging.
That said, even if it's out of scope, it would help to have a standardization of what a project needs and provides (to resolve dependencies), what version it has (to resolve updating needs) and how it can be tested.
Dependencies should be covered by existing metadata. Do you have specifics of why that's not sufficient?
Updating is likely out of scope - again, I don't know what you consider that to involve, but in my experience (which in this case is with non-Python projects) that covers database schema upgrading, migration of persistent object state, transparent upgrade of running sessions, etc. Which are hard problems, and to my knowledge not ones where there's even a well-recognised standard solution, much less something that the Python packaging community could formalise...
Further more, I think that ironically, Pypi.org would suffer the most from this tool.* namespace, since it'll need to implement all the "metadata" sections, "version" sections and custom-stuff sections from all tools to provide all the useful information it does now.
I have no idea what you mean by this - can you clarify? You seem to be saying that you foresee a problem with the current standards, but I don't understand what that is. None of the data in pyproject.toml is exposed by PyPI, so how is that affected?
So while it may not be relevant for my specific usecase, I still believe that PEP517's input data should be standardized inside py-project without the tool namespace.
That, we agree on. But someone needs to propose such a standard, and manage it through to approval.
For those tables to *not* lock people into your tool, you'd need other tools to agree to use the same names
No, I'm advocating that those sections be formalized (through new PEPs), so that tools SHALL use those sections for those bits of the PEP 517 api and we don't need 27 (random) different parses to obtain a dependency list for a given package.
I still think there's some confusion here. If by "package" you mean "distribution" or "project" (see https://packaging.python.org/glossary/) then dependency lists are part of the core metadata - https://packaging.python.org/specifications/core-metadata/ (specifically Requires-Dist). If, as I suspect, you mean it in another sense (likely an "application") then (a) please be careful not to misuse existing terminology if you want to avoid confusion, and (b) you're probably talking about requirements files. For that case, you should see https://packaging.python.org/discussions/install-requires-vs-requirements/ and https://caremad.io/posts/2013/07/setup-vs-requirement/. But basically no-one has yet proposed standardising the requirements file format, let alone migrating it to pyproject.toml (which personally I suspect is a bad fit for it). The nearest that anyone has come is the proposed pipfile format, which are I believe intended as a replacement for requirements.txt, but that's still at a relatively early stage of development as I understand it (there's certainly been no move to standardise).
Paul
On Tue, 11 Sep 2018 at 00:47, Paul Moore p.f.moore@gmail.com wrote:
On Mon, 10 Sep 2018 at 13:12, Melvyn Sopacua m.r.sopacua@gmail.com wrote:
PEP 517 extended the pyproject.toml format to add data for projects to define which build system they want to be used when they are built.
There's nothing yet for "building a project (not a library)". You could start with getting agreement on some terminology explaining what "a project (not a library)" is, and should be called :-) Would it cover pip? What about Mercurial? Ansible? A Django site? A blogging application backed by Django (or Flask, or whatever)? Maybe we need multiple terms...
Yes, that would sure help any standardization going forward here.
Great. That should keep you busy for a few months to start with :-) Seriously, we've touched on this in the past, and backed away from it because it's so hard to get a common agreement - I can't give specific references, and you may find it hard to locate anything in the archives as it's never really been a topic in its own right, more of a side issue, but don't assume it's straightforward.
The Python Packaging User Guide actually has a survey of "Not a library" packaging options now: https://packaging.python.org/overview/#packaging-python-applications
Daniel Holth had a pretty solid proposal years back for extending the wheel file format to cover more use cases: enhance the installation scheme directories to include the various GNU autoconf directories from https://www.gnu.org/prep/standards/html_node/Directory-Variables.html (perhaps with some further amendments enhancements to cope with different conventions on other operating systems).
Other than that, the starting point for deployment framework unification work would be to look at what the existing deployment frameworks (such as those mentioned under https://packaging.python.org/overview/#depending-on-a-framework ) take into account, and whether there's something that could be offered at the pyproject.toml level that would better enable portability of projects across different deployment tools.
Cheers, Nick.
participants (4)
-
Melvyn Sopacua -
Nick Coghlan -
Paul Moore -
Tzu-ping Chung