On Sun, Aug 7, 2016 at 4:15 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

On Aug 7, 2016, at 4:19 AM, Adi Roiban <adi@roiban.ro> wrote:

Hi,

I would like to remove twisted.python.dist as a public API without following the 1 year deprecation processes.

The module has a comment which informs people not to use it outside of Twisted, but I think it should be private module instead.

The ticket is here https://twistedmatrix.com/trac/ticket/8622 The PR is here https://github.com/twisted/twisted/pull/467 The branch is here https://github.com/twisted/twisted/tree/8622- setup-coverage

This is part of the setup.py cleanup effort for having most of the setup.py code outside of setup.py itself so that we can test it as part of the automated test suite.

twisted.python.dist3 was kept as a private API as it is already used by https://github.com/habnabit/twisted-depgraph

Please let me know if you are using twisted.python.dist outside of Twisted and are against this change or if you approve it.

Thanks!

+1 from me!

-1 from me. dist.py probably should have been a private API from the beginning, but it slipped through the cracks. However, what has been done is done. This change seems gratuitous and pointless with no benefits to anybody. -- Craig

On Sun, Aug 7, 2016 at 5:25 PM, Amber "Hawkie" Brown < hawkowl@atleastfornow.net> wrote:

On 8 Aug 2016, at 08:15, Craig Rodrigues <rodrigc@crodrigues.org> wrote:

On Sun, Aug 7, 2016 at 4:15 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

...

On Aug 7, 2016, at 4:19 AM, Adi Roiban <adi@roiban.ro> wrote:

Hi,

I would like to remove twisted.python.dist as a public API without following the 1 year deprecation processes.

The module has a comment which informs people not to use it outside of Twisted, but I think it should be private module instead.

The ticket is here https://twistedmatrix.com/trac/ticket/8622 The PR is here https://github.com/twisted/twisted/pull/467 The branch is here https://github.com/twisted/twisted/tree/8622-setup- coverage

This is part of the setup.py cleanup effort for having most of the setup.py code outside of setup.py itself so that we can test it as part of the automated test suite.

twisted.python.dist3 was kept as a private API as it is already used by https://github.com/habnabit/twisted-depgraph

Please let me know if you are using twisted.python.dist outside of Twisted and are against this change or if you approve it.

Thanks!

+1 from me!

-1 from me. dist.py probably should have been a private API from the beginning, but it slipped through the cracks. However, what has been done is done. This change seems gratuitous and pointless with no benefits to anybody.

-- Craig

I believe there's a point to it -- that is, we make it clear what is and what isn't user API. It's always said it in the header, but we should codify that it's really not user API. Plus it being private means we can make what changes we need for future setup.py changes, without deprecating things that nobody but us should ever use.

+1 from me.

I understand the motivation, but the time to have done this was when dist.py was originally reviewed. There are real problems in the Twisted code, and in the Twisted infrastructure. Focusing on something like this instead of those problems seems pointless, gratuitous and of little benefit. -- Craig

+1 from me On Mon, 8 Aug 2016 at 02:35 Craig Rodrigues <rodrigc@crodrigues.org> wrote:

I understand the motivation, but the time to have done this was when dist.py was originally reviewed.

There are real problems in the Twisted code, and in the Twisted infrastructure. Focusing on something like this instead of those problems seems pointless, gratuitous and of little benefit.

The problem is that if we defer this change until later, it becomes a lot more work then. If we do it now, while nobody is depending on the module, then we save ourselves that work in future by doing a trivial amount of work now.

On Aug 7, 2016, at 5:35 PM, Craig Rodrigues <rodrigc@crodrigues.org> wrote:

I understand the motivation, but the time to have done this was when dist.py was originally reviewed.

It certainly would have been better :-).

There are real problems in the Twisted code, and in the Twisted infrastructure.

This is also addressing a real problem. One of the major issues with Twisted is documentation noise. Developers trying to use Twisted to build things frequently complain about this: they look for an API to do XYZ, and they search the reference documentation, which produces dozens of confusing, irrelevant hits. There are multiple issues that interact here (see, for example, <https://github.com/twisted/pydoctor/issues/49> which is a big part of the problem) but one significant one is that our apparently-public API surface is just too big. Removing things like this, which are really _totally_ useless to anyone, is useful in and of itself and also amplifies the benefit of any other work on docs and tooling to properly segregate public and private API documentation. For a long time we were extremely conservative about doing any compatibility-breaking thing, even hypothetically breaking, even within the policy we'd set forth, and I think the result of that is a sprawling API that's incredibly hard to navigate, full of old junk that nobody should ever use. The slightly more aggressive attitude that has been exemplified by e.g. Amber has been a breath of fresh air when it comes to unburdening ourselves from literal garbage. So I think that "overburdened public API surface" is one of the largest problems facing Twisted and every little bit we can chip away at it is a real win. We do have to be super careful of course, we don't want to be chopping off obscure but useful pieces of API that programs were actually using, but that's what the compat policy (and judicious judgement) are for.

Focusing on something like this instead of those problems seems pointless, gratuitous and of little benefit.

In addition to underplaying the usefulness of maintenance like this, I think this comment belies acceptance of a superficially true falsehood - one that I've often fallen prey to myself - and that is, the idea that volunteer time is fungible. It isn't. If the members of a project have problems A, B, C, X, Y, Z in very clear ranked order of importance, and a volunteer ("V") shows up to work on Z, the impulse of the existing members of the project is "but you should really just work on 'A'!". V doesn't care about A, doesn't want to work on A, and the likely result of telling them they should be working on A is that they will feel bad about not working on A and their fix for Z will go unfinished. I'm not saying that necessarily applies in its full force here; after all Adi is a member of the project as well. But one of the corollaries of this realization is that if A were really so important someone would already be working on it :). We should totally have the ABCXYZ list in case anyone shows up just wondering what would be helpful to work on; but if someone shows up with a solution to Z, we should always try to incorporate it and help them feel motivated about continuing, hopefully up the list towards A of course, but maybe they are going to contribute stuff for M, N, and O that we didn't even realize was a problem. -glyph

The reason this module never had an underscore (and instead has the "don't use this outside of Twisted" line in the docstring) is because it originated in a time when Twisted was ostensibly split up into multiple projects, and I ill-advisedly decided that it technically needed to be "public" in order for the non-Twisted-core subprojects to use it. That was a silly decision, and given that the docstring has always told people not to use it, please feel free to rename it. -Chris On Mon, Aug 8, 2016, 1:56 AM Craig Rodrigues <rodrigc@crodrigues.org> wrote:

On Sun, Aug 7, 2016 at 11:01 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:

...

This is also addressing a real problem. One of the *major* issues with Twisted is *documentation noise*. Developers trying to use Twisted to build things frequently complain about this: they look for an API to do XYZ, and they search the reference documentation, which produces dozens of confusing, irrelevant hits. There are multiple issues that interact here (see, for example, <https://github.com/twisted/pydoctor/issues/49> which is a big part of the problem) but one significant one is that our apparently-public API surface is just too big. Removing things like this, which are really _totally_ useless to anyone, is useful in and of itself and also amplifies the benefit of any other work on docs and tooling to properly segregate public and private API documentation.

Making dist.py private seems to fall under the category of "code cleanliness", and doesn't seem to solve any specific pressing issue that I see. Reducing documentation noise is a good goal, but if dist.py was left the way it is, things would probably be OK with Twisted, and I think people could figure things out in Twisted with respect to the documentation.

Since you mention pydoctor, I will mention two issues, that I feel are more important to the project that moving around dist.py:

- pydoctor produces false errors, specifically in h2 code: https://buildbot.twistedmatrix.com/builders/documentation/builds/1291/steps/... - pydoctor doesn't work on Python 3: https://github.com/twisted/pydoctor/issues/96

I understand that Twisted is a volunteer project, and people can work on whatever they want. I also see on this list, that a few people support the dist.py change, so as long as Adi's change meets the Twisted coding standards, gets code review approval, and can pass all the buildbots, it can move forward, despite my objections.

-- Craig

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python