Twisted May 2020

twisted@python.org

19 participants
13 discussions

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing

by Glyph Lefkowitz

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

2 months, 1 week

[Twisted-Python] twistedchecker now uses pylint >= 2.4.4, can we use type annotations in Twisted now?

by Craig Rodrigues

Hi, Last week I did some work and updated twistedchecker so that it uses pylint >= 2.4.4. Before that, it was using a really old version of pylint. twistedchecker is run over the Twisted code as part of CI, and reports various style issues. Now that twistedchecker is using a newer pylint, it should be compatible with newer Python language features. Now that Twisted supports Python 3.5+ (with Python 2.7 support being dropped), is it OK to start using newer features of the Python language in the Twisted codebase such as type hints ( https://docs.python.org/3/library/typing.html )? I don't have experience with that feature, but it looks like it could be useful, when combined with a tool like mypy. Does anyone else have opinions on type hints and mypy? -- Craig

1 year, 3 months

[Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

by Craig Rodrigues

I would like to propose that Twisted drop support for Python 3.5 one year from now in 2021. Reasons: 1. Python 3.5.9 will be the final release of Python 3.5, in November 2019 ( https://www.python.org/dev/peps/pep-0478/ ) 2. Python 3.6.0 was released in December 2016 ( https://www.python.org/dev/peps/pep-0494/ ) 3. PyPy 7.1.1 introduced Python 3.6 support in April 2019 ( https://morepypy.blogspot.com/2019/04/pypy-711-bug-fix-release.html ). PyPy 7.3.1 is the latest PyPy version ( https://morepypy.blogspot.com/2020/04/pypy-731-released.html ). 4. Python 3.6 introduced support for variable annotations ( https://www.python.org/dev/peps/pep-0526/ ) 5. Python 3.6 introduced new string interpolation (f-string) support ( https://www.python.org/dev/peps/pep-0498/ ) 6. Python 3.6 introduced support for asynchronous generators ( https://www.python.org/dev/peps/pep-0525/ ) and asynchronous comprehensions ( https://www.python.org/dev/peps/pep-0530/ ) Based on item 2., Python 3.6.0 has been around for almost 4 years, which is a reasonable amount of time. Based on item 3., PyPy supports Python 3.6. It might be nice to take advantage of some of the language features introduces in items 4-6. I think dropping support one year from now in 2021, would give sufficient notice, and be in line with the Twisted Compatibility Policy ( https://twistedmatrix.com/documents/current/core/development/policy/compati… ). None of these items are critial, but I thought I would ask, to get people's opinions. -- Craig

1 year, 4 months

[Twisted-Python] Deprecate and remove t.internet.ssl.DistinguishedName ?

by Wim Lewis

I'm looking at a fix for bug <https://twistedmatrix.com/trac/ticket/9804> (Cannot load a PEM certificate with Unicode in subject). The underlying problem is that the DistinguishedName class can't handle non-ascii AVAs. The fix I've made simply avoids creating DistinguishedName instances when it isn't necessary, but that leaves the question of what to do with the class. I think that the best thing to do is to deprecate the class entirely and replace it with simpler API. Reasons I think that the DN class is broken: - The values in a certificate are conceptually text-strings, not byte strings; they may be in ASCII, UTF8, UTF16, or several other encodings. However - DN represents these textual values as `bytes` instead of `str` - DN can't handle non-ASCII-representable values at all, even if the user never tries to access that value - It can only handle a subset of the attribute-assertions found in a PKIX DN; there's no escape hatch for others (e.g. OID keys or whatever) - It can't represent the full structure of a DN (specific ordering, multiple-value RDNs, AVAs whos values aren't textual, etc.) --- these are not common in the PKIX world but they are valid What I propose as an alternative: - Replace APIs that take `DistinguishedName` classes with ones that take `Union[OpenSSL.crypto.X509Name, dict]` where the `dict` format is parsed with the same convenience semantics as DistinguishedName, except that values are `str` - Replace APIs that return `DistinguishedName` with ones that return OpenSSL.crypto.X509Name, which is already fairly convenient to use (e.g. it has attributes for retrieving/setting commonName and so on without dealing with the full complexity of X.500 names) - Deprecate `DistinguishedName` and the APIs that use it for eventual removal - Expose a convenience function for the dict -> X509Name transform Any objections? Thoughts on how I should go about doing this? Should I do it as part of this Trac ticket or split it out? The only downside I can think of is that this exposes the OpenSSL.crypto.X509Name type as part of Twisted's API. I don't think this is a huge reduction in flexibility --- Twisted's API already somewhat assumes that TLS is implemented using OpenSSL, and only users whose needs are *already* not well met by DistinguishedName will care if that `Union` type changes in the future. Wim Lewis / wiml(a)hhhh.org

1 year, 4 months

[Twisted-Python] Weekly Bug Summary

by trac＠twistedmatrix.com

1 year, 4 months

[Twisted-Python] Weekly Bug Summary

by trac＠twistedmatrix.com

1 year, 4 months

Re: [Twisted-Python] SMB server component for twisted

by Glyph

> On May 14, 2020, at 5:23 PM, Wilfredo Sánchez Vega <wsanchez@wsanchez.net(mailto:wsanchez@wsanchez.net)> wrote: > On May 7, 2020, at 12:48 AM, Glyph <glyph@twistedmatrix.com(mailto:glyph@twistedmatrix.com)> wrote: > > > > If you want to include it in Twisted itself, your best bet is to actually develop it within twisted, as a series of small contributions, rather than as one gigantic one-shot one. Contributions over, say, 400 lines, take exponentially longer to review. > > > > Developing it within Twisted will make things go slower; you'll need to get everything code reviewed, you'll need to support multiple versions of Python (no py2 any more, but py3.5 is still pretty old), you'll have to have full test coverage from the get-go. But doing these things from the start is much easier than trying to retrofit them. > > > > I actually think that this would be a pretty good fit for Twisted, in the same way that it's been a benefit to have Conch maintained alongside the rest of Twisted. I can see you're developing things very much in line with Twisted's architecture (using cred for authentication, a realm interface, etc) and you've voiced this interest, so it would be great to have you along! > I think it's great to get an SMB implementation in the Twisted org, but why would we even consider adding something like this to the main Twisted project? Just practically speaking, we've tried both approaches, and (very broadly generalizing) this one works and the other one doesn't. Separate projects that have been adopted from outside after they gained some traction are doing okay (i.e. Treq and Klein, some of the most popular applications for Twisted). But others (txacme) are suffering from a lack of maintenance and still others (ldaptor) just have a largely non-overlapping community and hardly benefit at all from org membership. And as you've noted, even the ones doing their best often want for reviews for far longer than Twisted's own latency. > The main repo is large, and including Twisted in your project brings along far more functionality than you are likely to use, and packages are a fine way to get the functionality you want, so what's the logic for including a new thing in Twisted proper? Abstractly you make a plausible case. Concretely we tried to start splitting things up on the axes of protocol support and it just made a giant mess and created lots of overhead for the project. Modern infrastructure makes this slightly easier (last time we tried to start this process we didn't have a Github or a Travis), but not much easier. And even modern, better infrastructure leads to an avalanche of tedium as we copy CI configuration back and forth endlessly, replicating lessons that we learn about travis, and azure, and codecov, and (ad nauseam) Even in the abstract case though there’s a question of what twisted “is” and on what axes it should be split up. I think splitting up by protocol was largely a mistake, but splitting some of the really low level core stuff out (towncrier, automat, and hopefully filepath and deferred) has been pretty successful and worthwhile. Particularly in a post-asyncio world, the core event loop isn’t super interesting and the broad protocol support is the appeal. So even in the abstract case, there’s a case to be made that twisted is a bunch of implementations and interfaces for wire protocols that involve navigating hierarchies of things (DNS, HTTP, IMAP, SFTP) and SMB fits right in there. > You cite Conch above, where I'd actually argue that Conch should be moved to a separate repo. Am I crazy? Not crazy, there's a valid case to be made, but Conch has received a tremendous amount of routine maintenance keeping it up to date with Twisted and Python API evolution by members of the project who would almost certainly otherwise not bother, just to keep the test suite going and the project as a whole integrated. This allows it to remain viable for the much smaller number of SSH-specific contributions that come along. > That said, a counter argument is that projects in the Twisted org suffer greatly from poor participation and strict development rules, resulting in glacial progress. It's not unusual for me to have a PR take 6 months to a year to get a review (I have two right now that have been waiting since November), and perhaps such PRs would get some attention in the main repo. But I think that's an unfortunate way to address that problem. This is just a natural consequence of affordances provided by the tooling we presently have. Personally I look at the global queue of pull requests across all these projects and it's kind of exhausting. I don't think many other people do. Case in point: how many ldaptor or txacme reviews have you done since November? This is not to throw any shade; I’m not saying you should have, but it illustrates that right now, separate repos engender a really unfortunate level of community fragmentation. I agree that this is not really an ideal situation, but addressing it requires building a degree of tooling that we just don’t have the resources to do. Our previous attempt to do so was … bad; it kind of illustrated that we don’t have the relevant expertise to build these kinds of tools and mores either, and we just got kind of lucky with the somewhat random pile of jank we have landed on with the current twistedmatrix.com process. Anyway if someone wanted to do the work to separate out the concerns of version control, pip installability, issue tracking, and integrated regression testing for a whole suite of related python packages in an org, I would be extremely excited, but the benefits of just having Twisted be, say, 5% bigger overall to foster development of a whole new popular and intimidating protocol seem clear given the state of the world today. If those tools do show up later to make this tractable, we can easily split it out later! Along with conch, and mail, and names, for that matter… -glyph

1 year, 4 months

[Twisted-Python] Weekly Bug Summary

by trac＠twistedmatrix.com

1 year, 5 months

[Twisted-Python] SMB server component for twisted

by Ian Haywood

I have begun work on a SMB (Server Message Block; Windows filesharing) server protocol for twisted. Work so far is here: https://github.com/ihaywood3/twsmb I'm looking for any advice particularly around what I should be doing so this code is suitable for inclusion in twisted. Ian Haywood

1 year, 5 months

[Twisted-Python] Weekly Bug Summary

by trac＠twistedmatrix.com

1 year, 5 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

Twisted May 2020