[Twisted-Python] Twisted 16.7.0rc1 Release Candidate Announcement
Hello everyone, and happy holidays! Although it didn't quite fall on the solstice, one could call this Twisted release a little ray of sunshine, with some nice new improvements. 16.7.0rc1 brings the following stocking stuffers: - twisted.web.client.Agent now supports IPv6! It's also now the primary web client in Twisted, with twisted.web.client.getPage being deprecated in favour of it and Treq. - twisted.internet.ssl.CertificateOptions has had its `method` argument deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments. This allows you to better give a range of versions of TLS you wish to negotiate, rather than forcing yourself to any one version. - twisted.internet.ssl.CertificateOptions will use OpenSSL's MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by idle TLS connections. - You can now call the new twist runner with `python -m twisted`. - Conch has the beginnings of ECDH key exchange, and supports `hmac-sha2-384` in twisted.conch.ssh. - Better Unicode support in twisted.internet.reactor.spawnProcess, especially on Windows on Python 3.6. - More Python 3 porting in Conch, and more under-the-hood changes to facilitate a Twisted-wide jump to new-style classes only on Python 2 in 2018/2019. This release has also been tested on Python 3.6 on Linux. - Lots of deprecated code removals, to make a sleeker, less confusing Twisted. - 20+ closed tickets. You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ <https://twistedmatrix.com/Releases/rc/16.7.0rc1/> , or you can try it out from PyPI: python -m pip install Twisted==16.7.0rc1 Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year. With festive hoots, Amber Brown
On 2016-12-23, at 4:33, Amber Hawkie Brown <hawkowl@atleastfornow.net> wrote:
[...]
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber, Do you think https://github.com/twisted/twisted/pull/599 can go in? As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved. Thanks and Happy Holidays, -- exvito
On Dec 23, 2016, at 3:49 AM, ex vito <ex.vitorino@gmail.com> wrote:
On 2016-12-23, at 4:33, Amber Hawkie Brown <hawkowl@atleastfornow.net <mailto:hawkowl@atleastfornow.net>> wrote:
[...]
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ <https://twistedmatrix.com/Releases/rc/16.7.0rc1/> , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 <https://github.com/twisted/twisted/pull/599> can go in? As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017). However, I have to say I'm very sorry this did not get merged in a timely way! It seems I didn't get the github notification when you @mentioned me; I have no idea why that would be. I'll get test results on trunk and get it merged, and it should be in the next 17.x release. -glyph
On Dec 23, 2016, at 12:31 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Dec 23, 2016, at 3:49 AM, ex vito <ex.vitorino@gmail.com <mailto:ex.vitorino@gmail.com>> wrote:
On 2016-12-23, at 4:33, Amber Hawkie Brown <hawkowl@atleastfornow.net <mailto:hawkowl@atleastfornow.net>> wrote:
[...]
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ <https://twistedmatrix.com/Releases/rc/16.7.0rc1/> , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 <https://github.com/twisted/twisted/pull/599> can go in? As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017).
However, I have to say I'm very sorry this did not get merged in a timely way! It seems I didn't get the github notification when you @mentioned me; I have no idea why that would be.
I'll get test results on trunk and get it merged, and it should be in the next 17.x release.
-glyph
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5 <https://twistedmatrix.com/trac/ticket/8911#comment:5> On a related note - this still linked the '#' syntax back to the ticket number in Trac. Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :). -glyph
On 2016-12-23, at 21:22, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Dec 23, 2016, at 12:31 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Dec 23, 2016, at 3:49 AM, ex vito <ex.vitorino@gmail.com> wrote:
On 2016-12-23, at 4:33, Amber Hawkie Brown <hawkowl@atleastfornow.net> wrote:
[...]
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 can go in? As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017).
However, I have to say I'm very sorry this did not get merged in a timely way! It seems I didn't get the github notification when you @mentioned me; I have no idea why that would be.
I'll get test results on trunk and get it merged, and it should be in the next 17.x release.
Glyph, The next 17.x release sounds good. I'm in no hurries and that keeps an open time window to address the very tightly related https://twistedmatrix.com/trac/ticket/8912 which I recently noted was created by Tom Prince.
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).
Thanks for the quick feedback and heads up on the '#' link. Regards, -- exvito
On Dec 24, 2016, at 6:01 AM, ex vito <ex.vitorino@gmail.com> wrote:
On 2016-12-23, at 21:22, Glyph Lefkowitz <glyph@twistedmatrix.com <mailto:glyph@twistedmatrix.com>> wrote:
On Dec 23, 2016, at 12:31 PM, Glyph Lefkowitz <glyph@twistedmatrix.com <mailto:glyph@twistedmatrix.com>> wrote:
On Dec 23, 2016, at 3:49 AM, ex vito <ex.vitorino@gmail.com <mailto:ex.vitorino@gmail.com>> wrote:
On 2016-12-23, at 4:33, Amber Hawkie Brown <hawkowl@atleastfornow.net <mailto:hawkowl@atleastfornow.net>> wrote:
[...]
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ <https://twistedmatrix.com/Releases/rc/16.7.0rc1/> , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
Amber,
Do you think https://github.com/twisted/twisted/pull/599 <https://github.com/twisted/twisted/pull/599> can go in? As far as I can tell, it's been reviewed and approved for merging. If not, please point out what should be improved.
Amber is the release manager and therefore has final say, but, generally, we try not to roll more things into releases. Instead, we try to roll releases more frequently (which has been a resounding success during 2016, and hopefully we can keep that momentum during 2017).
However, I have to say I'm very sorry this did not get merged in a timely way! It seems I didn't get the github notification when you @mentioned me; I have no idea why that would be.
I'll get test results on trunk and get it merged, and it should be in the next 17.x release.
Glyph,
The next 17.x release sounds good. I'm in no hurries and that keeps an open time window to address the very tightly related https://twistedmatrix.com/trac/ticket/8912 <https://twistedmatrix.com/trac/ticket/8912> which I recently noted was created by Tom Prince.
Great. Basically, try not to worry about releases too much - just do the work as quick as you can, do reviews to speed along the queue in general, and the ever-faster release cadence that Amber has put in place will catch your changes soon enough.
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5 <https://twistedmatrix.com/trac/ticket/8911#comment:5>
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).
Thanks for the quick feedback and heads up on the '#' link.
This was a heads-up for Jean-Paul, who has been trying to change this behavior (to make '#' on new commits refer to Github issues / PRs, and 'ticket:' refer to Trac tickets, so we don't get spurious email notifications about old Trac tickets). You can pretty much ignore it :). -glyph
On Fri, Dec 23, 2016 at 4:22 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).
-glyph
It looks like the link to ticket 599 in the merge/resolve comment on ticket 8911 is a different kind of link than the one I was looking in to. I agree it would be nice to fix this, too, and perhaps I'll take a look. My goal was only to stop ticket 599 (for example) from also receiving a copy of this comment - which appears to have succeeded. Let me know if I've missed something. Jean-Paul
On Dec 26, 2016, at 16:14, Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote:
On Fri, Dec 23, 2016 at 4:22 PM, Glyph Lefkowitz <glyph@twistedmatrix.com <mailto:glyph@twistedmatrix.com>> wrote:
It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5 <https://twistedmatrix.com/trac/ticket/8911#comment:5>
On a related note - this still linked the '#' syntax back to the ticket number in Trac.
Perhaps related - tomprince fixed the bugs that were preventing us from upgrading us to 1.2, and upgraded us to 1.2. So perhaps this has introduced a problem :).
-glyph
It looks like the link to ticket 599 in the merge/resolve comment on ticket 8911 is a different kind of link than the one I was looking in to. I agree it would be nice to fix this, too, and perhaps I'll take a look. My goal was only to stop ticket 599 (for example) from also receiving a copy of this comment - which appears to have succeeded. Let me know if I've missed something.
Sorry - this is fine. I mistakenly assumed that the fact that the link was highlighted meant that the comment happened. If we were to kill '#' links, that would also break all the existing commit messages, which would be bad. So please leave it as-is! -glyph
Seems to require a new Python module, Automat. (Not mentioned in the NEWS file, INSTALL, etc.) I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format. Downloaded it, but I'm not sure what to do next. I was able to get it working by downloading and installing get-pip.py and Automat (contrary to the python.org documentation, I didn't have pip installed on my system even though it is Python 2.7.13 downloaded from python.org.) sudo python get-pip.py sudo pip install Automat This is worrisome to me because I think this requires Internet access to pypi.python.org, which I have on my development and testing systems, but not on my production systems. Everything else I've ever needed, I could download and install source files. Having done this, my app seems to run fine (only quick sanity testing so far.) ----install--- Searching for Automat>=0.3.0 Reading https://pypi.python.org/simple/Automat/ Download error on https://pypi.python.org/simple/Automat/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found! Couldn't find index page for 'Automat' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading https://pypi.python.org/simple/ Download error on https://pypi.python.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found! No local packages or working download links found for Automat>=0.3.0 error: Could not find suitable distribution for Requirement.parse('Automat>=0.3.0') ------ ---try to use without Automat--- medusa@Medusa:~/run> python medusa_server.py Rain Traceback (most recent call last): File "medusa_server.py", line 72, in <module> from twisted.application import service, internet File "/usr/lib/python2.7/site-packages/Twisted-16.7.0rc1-py2.7-linux-x86_64.egg/twisted/application/internet.py", line 54, in <module> from automat import MethodicalMachine ImportError: No module named automat ------ On 12/22/2016 11:33 PM, Amber "Hawkie" Brown wrote:
Hello everyone, and happy holidays!
Although it didn't quite fall on the solstice, one could call this Twisted release a little ray of sunshine, with some nice new improvements. 16.7.0rc1 brings the following stocking stuffers:
- twisted.web.client.Agent now supports IPv6! It's also now the primary web client in Twisted, with twisted.web.client.getPage being deprecated in favour of it and Treq. - twisted.internet.ssl.CertificateOptions has had its `method` argument deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments. This allows you to better give a range of versions of TLS you wish to negotiate, rather than forcing yourself to any one version. - twisted.internet.ssl.CertificateOptions will use OpenSSL's MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by idle TLS connections. - You can now call the new twist runner with `python -m twisted`. - Conch has the beginnings of ECDH key exchange, and supports `hmac-sha2-384` in twisted.conch.ssh. - Better Unicode support in twisted.internet.reactor.spawnProcess, especially on Windows on Python 3.6. - More Python 3 porting in Conch, and more under-the-hood changes to facilitate a Twisted-wide jump to new-style classes only on Python 2 in 2018/2019. This release has also been tested on Python 3.6 on Linux. - Lots of deprecated code removals, to make a sleeker, less confusing Twisted. - 20+ closed tickets.
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
With festive hoots, Amber Brown
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539
Hi John, .whl is just an optimised binary format, and can be installed by pip from a local file. If you want to get all the dependencies of Twisted in a future-proof way, without network on production, just do this on a machine with the same arch/OS: hawkowl /p/t/tx> pip download twisted[tls]==16.7.0rc1 [LOADING BARS ELINED] Successfully downloaded twisted zope.interface constantly incremental Automat pyopenssl service-identity idna setuptools six characteristic cryptography pyasn1 pyasn1-modules attrs ipaddress enum34 cffi pycparser hawkowl /p/t/tx> ls Automat-0.3.0-py2.py3-none-any.whl ipaddress-1.0.17-py2-none-any.whl Twisted-16.7.0rc1.tar.bz2 pyOpenSSL-16.2.0-py2.py3-none-any.whl attrs-16.3.0-py2.py3-none-any.whl pyasn1-0.1.9-py2.py3-none-any.whl cffi-1.9.1-cp27-cp27m-macosx_10_10_intel.whl pyasn1_modules-0.0.8-py2.py3-none-any.whl characteristic-14.3.0-py2.py3-none-any.whl pycparser-2.17.tar.gz constantly-15.1.0-py2.py3-none-any.whl service_identity-16.0.0-py2.py3-none-any.whl cryptography-1.7.1-cp27-cp27m-macosx_10_10_intel.whl setuptools-32.3.0-py2.py3-none-any.whl enum34-1.1.6-py2-none-any.whl six-1.10.0-py2.py3-none-any.whl idna-2.2-py2.py3-none-any.whl zope.interface-4.3.3-cp27-cp27m-macosx_10_11_x86_64.whl incremental-16.10.1-py2.py3-none-any.whl Put these files on the target machine. Then on the target machine, with no internet connection: hawkowl /p/t/tx> virtualenv venv Using real prefix '/System/Library/Frameworks/Python.framework/Versions/2.7' New python executable in /private/tmp/tx/venv/bin/python Installing setuptools, pip, wheel...done. # Because incremental is setup_requires on twisted, install it explicitly or setuptools will go try and fetch it hawkowl /p/t/tx [1]> venv/bin/pip install --no-index --find-links ./ incremental Collecting incremental Installing collected packages: incremental Successfully installed incremental-16.10.1 hawkowl /p/t/tx> venv/bin/pip install --no-index --find-links ./ twisted[tls] Collecting twisted[tls] Collecting Automat>=0.3.0 (from twisted[tls]) Collecting zope.interface>=3.6.0 (from twisted[tls]) Collecting constantly>=15.1 (from twisted[tls]) Requirement already satisfied: incremental>=16.10.1 in ./venv/lib/python2.7/site-packages (from twisted[tls]) Collecting service-identity; extra == "tls" (from twisted[tls]) Collecting idna>=0.6; extra == "tls" (from twisted[tls]) Collecting pyopenssl>=16.0.0; extra == "tls" (from twisted[tls]) Requirement already satisfied: six in ./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls]) Requirement already satisfied: characteristic in ./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls]) Requirement already satisfied: setuptools in ./venv/lib/python2.7/site-packages (from zope.interface>=3.6.0->twisted[tls]) Collecting pyasn1 (from service-identity; extra == "tls"->twisted[tls]) Collecting pyasn1-modules (from service-identity; extra == "tls"->twisted[tls]) Requirement already satisfied: attrs in ./venv/lib/python2.7/site-packages (from service-identity; extra == "tls"->twisted[tls]) Collecting cryptography>=1.3.4 (from pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting ipaddress (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting enum34 (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting pycparser (from cffi>=1.4.1->cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Installing collected packages: Automat, zope.interface, constantly, pyasn1, pyasn1-modules, idna, ipaddress, enum34, pycparser, cffi, cryptography, pyopenssl, service-identity, twisted Successfully installed Automat-0.3.0 cffi-1.9.1 constantly-15.1.0 cryptography-1.7.1 enum34-1.1.6 idna-2.2 ipaddress-1.0.17 pyasn1-0.1.9 pyasn1-modules-0.0.8 pycparser-2.17 pyopenssl-16.2.0 service-identity-16.0.0 twisted-16.7.0rc1 zope.interface-4.3.3 This will install Twisted and its dependencies the official pip way, without needing to hardcode which libraries you fetch, and will do the right thing as Twisted eventually adds more dependencies (like when we split out FilePath and Deferred). - Amber
On 29 Dec. 2016, at 10:20, John Santos <john@egh.com> wrote:
Seems to require a new Python module, Automat. (Not mentioned in the NEWS file, INSTALL, etc.) I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format. Downloaded it, but I'm not sure what to do next. I was able to get it working by downloading and installing get-pip.py and Automat (contrary to the python.org documentation, I didn't have pip installed on my system even though it is Python 2.7.13 downloaded from python.org.) sudo python get-pip.py sudo pip install Automat
This is worrisome to me because I think this requires Internet access to pypi.python.org, which I have on my development and testing systems, but not on my production systems. Everything else I've ever needed, I could download and install source files.
Having done this, my app seems to run fine (only quick sanity testing so far.)
----install--- Searching for Automat>=0.3.0 Reading https://pypi.python.org/simple/Automat/ <https://pypi.python.org/simple/Automat/> Download error on https://pypi.python.org/simple/Automat/ <https://pypi.python.org/simple/Automat/>: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found! Couldn't find index page for 'Automat' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading https://pypi.python.org/simple/ <https://pypi.python.org/simple/> Download error on https://pypi.python.org/simple/ <https://pypi.python.org/simple/>: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found! No local packages or working download links found for Automat>=0.3.0 error: Could not find suitable distribution for Requirement.parse('Automat>=0.3.0') ------
---try to use without Automat--- medusa@Medusa:~/run> python medusa_server.py Rain Traceback (most recent call last): File "medusa_server.py", line 72, in <module> from twisted.application import service, internet File "/usr/lib/python2.7/site-packages/Twisted-16.7.0rc1-py2.7-linux-x86_64.egg/twisted/application/internet.py", line 54, in <module> from automat import MethodicalMachine ImportError: No module named automat ------
On 12/22/2016 11:33 PM, Amber "Hawkie" Brown wrote:
Hello everyone, and happy holidays!
Although it didn't quite fall on the solstice, one could call this Twisted release a little ray of sunshine, with some nice new improvements. 16.7.0rc1 brings the following stocking stuffers:
- twisted.web.client.Agent now supports IPv6! It's also now the primary web client in Twisted, with twisted.web.client.getPage being deprecated in favour of it and Treq. - twisted.internet.ssl.CertificateOptions has had its `method` argument deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments. This allows you to better give a range of versions of TLS you wish to negotiate, rather than forcing yourself to any one version. - twisted.internet.ssl.CertificateOptions will use OpenSSL's MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by idle TLS connections. - You can now call the new twist runner with `python -m twisted`. - Conch has the beginnings of ECDH key exchange, and supports `hmac-sha2-384` in twisted.conch.ssh. - Better Unicode support in twisted.internet.reactor.spawnProcess, especially on Windows on Python 3.6. - More Python 3 porting in Conch, and more under-the-hood changes to facilitate a Twisted-wide jump to new-style classes only on Python 2 in 2018/2019. This release has also been tested on Python 3.6 on Linux. - Lots of deprecated code removals, to make a sleeker, less confusing Twisted. - 20+ closed tickets.
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ <https://twistedmatrix.com/Releases/rc/16.7.0rc1/> , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
With festive hoots, Amber Brown
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com <mailto:Twisted-Python@twistedmatrix.com> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python <http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python>
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539 _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Thank you (and Glyph for his followup) for the detailed instructions. I will be trying this on our 2nd development system in a few days (or possibly sooner on the test system if 16.7.0 is released before I get around to it :-)) and will let you know if I encounter any difficulties. On 12/28/2016 6:37 PM, Amber "Hawkie" Brown wrote:
Hi John,
.whl is just an optimised binary format, and can be installed by pip from a local file. If you want to get all the dependencies of Twisted in a future-proof way, without network on production, just do this on a machine with the same arch/OS:
hawkowl /p/t/tx> pip download twisted[tls]==16.7.0rc1 [LOADING BARS ELINED] Successfully downloaded twisted zope.interface constantly incremental Automat pyopenssl service-identity idna setuptools six characteristic cryptography pyasn1 pyasn1-modules attrs ipaddress enum34 cffi pycparser
hawkowl /p/t/tx> ls Automat-0.3.0-py2.py3-none-any.whl ipaddress-1.0.17-py2-none-any.whl Twisted-16.7.0rc1.tar.bz2 pyOpenSSL-16.2.0-py2.py3-none-any.whl attrs-16.3.0-py2.py3-none-any.whl pyasn1-0.1.9-py2.py3-none-any.whl cffi-1.9.1-cp27-cp27m-macosx_10_10_intel.whl pyasn1_modules-0.0.8-py2.py3-none-any.whl characteristic-14.3.0-py2.py3-none-any.whl pycparser-2.17.tar.gz constantly-15.1.0-py2.py3-none-any.whl service_identity-16.0.0-py2.py3-none-any.whl cryptography-1.7.1-cp27-cp27m-macosx_10_10_intel.whl setuptools-32.3.0-py2.py3-none-any.whl enum34-1.1.6-py2-none-any.whl six-1.10.0-py2.py3-none-any.whl idna-2.2-py2.py3-none-any.whl zope.interface-4.3.3-cp27-cp27m-macosx_10_11_x86_64.whl incremental-16.10.1-py2.py3-none-any.whl
Put these files on the target machine. Then on the target machine, with no internet connection:
hawkowl /p/t/tx> virtualenv venv Using real prefix '/System/Library/Frameworks/Python.framework/Versions/2.7' New python executable in /private/tmp/tx/venv/bin/python Installing setuptools, pip, wheel...done.
# Because incremental is setup_requires on twisted, install it explicitly or setuptools will go try and fetch it hawkowl /p/t/tx [1]> venv/bin/pip install --no-index --find-links ./ incremental Collecting incremental Installing collected packages: incremental Successfully installed incremental-16.10.1
hawkowl /p/t/tx> venv/bin/pip install --no-index --find-links ./ twisted[tls] Collecting twisted[tls] Collecting Automat>=0.3.0 (from twisted[tls]) Collecting zope.interface>=3.6.0 (from twisted[tls]) Collecting constantly>=15.1 (from twisted[tls]) Requirement already satisfied: incremental>=16.10.1 in ./venv/lib/python2.7/site-packages (from twisted[tls]) Collecting service-identity; extra == "tls" (from twisted[tls]) Collecting idna>=0.6; extra == "tls" (from twisted[tls]) Collecting pyopenssl>=16.0.0; extra == "tls" (from twisted[tls]) Requirement already satisfied: six in ./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls]) Requirement already satisfied: characteristic in ./venv/lib/python2.7/site-packages (from Automat>=0.3.0->twisted[tls]) Requirement already satisfied: setuptools in ./venv/lib/python2.7/site-packages (from zope.interface>=3.6.0->twisted[tls]) Collecting pyasn1 (from service-identity; extra == "tls"->twisted[tls]) Collecting pyasn1-modules (from service-identity; extra == "tls"->twisted[tls]) Requirement already satisfied: attrs in ./venv/lib/python2.7/site-packages (from service-identity; extra == "tls"->twisted[tls]) Collecting cryptography>=1.3.4 (from pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting ipaddress (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting enum34 (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Collecting pycparser (from cffi>=1.4.1->cryptography>=1.3.4->pyopenssl>=16.0.0; extra == "tls"->twisted[tls]) Installing collected packages: Automat, zope.interface, constantly, pyasn1, pyasn1-modules, idna, ipaddress, enum34, pycparser, cffi, cryptography, pyopenssl, service-identity, twisted Successfully installed Automat-0.3.0 cffi-1.9.1 constantly-15.1.0 cryptography-1.7.1 enum34-1.1.6 idna-2.2 ipaddress-1.0.17 pyasn1-0.1.9 pyasn1-modules-0.0.8 pycparser-2.17 pyopenssl-16.2.0 service-identity-16.0.0 twisted-16.7.0rc1 zope.interface-4.3.3
This will install Twisted and its dependencies the official pip way, without needing to hardcode which libraries you fetch, and will do the right thing as Twisted eventually adds more dependencies (like when we split out FilePath and Deferred).
- Amber
On 29 Dec. 2016, at 10:20, John Santos <john@egh.com <mailto:john@egh.com>> wrote:
Seems to require a new Python module, Automat. (Not mentioned in the NEWS file, INSTALL, etc.) I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format. Downloaded it, but I'm not sure what to do next. I was able to get it working by downloading and installing get-pip.py and Automat (contrary to the python.org <http://python.org> documentation, I didn't have pip installed on my system even though it is Python 2.7.13 downloaded from python.org <http://python.org>.) sudo python get-pip.py sudo pip install Automat
This is worrisome to me because I think this requires Internet access to pypi.python.org <http://pypi.python.org>, which I have on my development and testing systems, but not on my production systems. Everything else I've ever needed, I could download and install source files.
Having done this, my app seems to run fine (only quick sanity testing so far.)
----install--- Searching for Automat>=0.3.0 Reading https://pypi.python.org/simple/Automat/ Download error on https://pypi.python.org/simple/Automat/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found! Couldn't find index page for 'Automat' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading https://pypi.python.org/simple/ Download error on https://pypi.python.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) -- Some packages may not be found! No local packages or working download links found for Automat>=0.3.0 error: Could not find suitable distribution for Requirement.parse('Automat>=0.3.0') ------
---try to use without Automat--- medusa@Medusa:~/run> python medusa_server.py Rain Traceback (most recent call last): File "medusa_server.py", line 72, in <module> from twisted.application import service, internet File "/usr/lib/python2.7/site-packages/Twisted-16.7.0rc1-py2.7-linux-x86_64.egg/twisted/application/internet.py", line 54, in <module> from automat import MethodicalMachine ImportError: No module named automat ------
On 12/22/2016 11:33 PM, Amber "Hawkie" Brown wrote:
Hello everyone, and happy holidays!
Although it didn't quite fall on the solstice, one could call this Twisted release a little ray of sunshine, with some nice new improvements. 16.7.0rc1 brings the following stocking stuffers:
- twisted.web.client.Agent now supports IPv6! It's also now the primary web client in Twisted, with twisted.web.client.getPage being deprecated in favour of it and Treq. - twisted.internet.ssl.CertificateOptions has had its `method` argument deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo, and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments. This allows you to better give a range of versions of TLS you wish to negotiate, rather than forcing yourself to any one version. - twisted.internet.ssl.CertificateOptions will use OpenSSL's MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by idle TLS connections. - You can now call the new twist runner with `python -m twisted`. - Conch has the beginnings of ECDH key exchange, and supports `hmac-sha2-384` in twisted.conch.ssh. - Better Unicode support in twisted.internet.reactor.spawnProcess, especially on Windows on Python 3.6. - More Python 3 porting in Conch, and more under-the-hood changes to facilitate a Twisted-wide jump to new-style classes only on Python 2 in 2018/2019. This release has also been tested on Python 3.6 on Linux. - Lots of deprecated code removals, to make a sleeker, less confusing Twisted. - 20+ closed tickets.
You can get the tarball and the NEWS file at https://twistedmatrix.com/Releases/rc/16.7.0rc1/ , or you can try it out from PyPI:
python -m pip install Twisted==16.7.0rc1
Please test it, and let me know how your applications fare, good or bad! If nothing comes up, 16.7 looks like it'll land just before the new year.
With festive hoots, Amber Brown
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539 _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com <mailto:Twisted-Python@twistedmatrix.com> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539
On Dec 28, 2016, at 15:20, John Santos <JOHN@egh.com> wrote:
Seems to require a new Python module, Automat.
Good catch, that probably should have been announced at least a little :).
(Not mentioned in the NEWS file, INSTALL, etc.)
Despite the fact that it would have been good to mention this particular dependency because its adoption is a new architectural direction for Twisted, in general we're actually planning to do less communication of dependencies in this way. Right now, we have redundant listing of dependencies in a few places, but in the future, setup.py (or more specifically, "packaging metadata") will be the one, authoritative place that such things are listed.
I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format.
Although you probably should just use the wheel, the fact that there was only a wheel was an oversight. I've uploaded an sdist and pushed a corresponding tag (after making sure that the source files all matched, of course).
Downloaded it, but I'm not sure what to do next. [...] This is worrisome to me because I think this requires Internet access to pypi.python.org, which I have on my development and testing systems, but not on my production systems. Everything else I've ever needed, I could download and install source files.
As Amber already explained, the ability to deploy offline is very much an important part of the pip/wheel ecosystem. I went into some detail in one of my own blog posts on this: https://glyph.twistedmatrix.com/2015/03/docker-deploy-double-dutch.html <https://glyph.twistedmatrix.com/2015/03/docker-deploy-double-dutch.html> Increasingly, we are assuming a pip-like packaging toolchain for dealing with Twisted's dependencies, so getting familiar with this stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile. (It'll make your life easier in more ways than one.) It should handily meet all your requirements for security and flexibility, and if it doesn't, we have a good working relationship with the maintainers of all those tools so we'd like to hear about if it's a problem :).
Having done this, my app seems to run fine (only quick sanity testing so far.)
Thanks for testing! -glyph
On Thu, Dec 29, 2016 at 05:27:44AM -0800, Glyph Lefkowitz wrote:
Increasingly, we are assuming a pip-like packaging toolchain for dealing with Twisted's dependencies, so getting familiar with this stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile. (It'll make your life easier in more ways than one.)
I wasn't going to mention it (because we found a workaround), but since the topic has come up... My current employer has a fully pip-based packaging toolchain for all our internal Python apps, where we build wheels for our apps and all their dependencies, and then deploy them into a virtualenv in production. For reproducability reasons, we have a PyPI mirror inside the corporate firewall, and our wheel-building Docker image has a `pip.conf` configured to point at it. Twisted 16.6 broke this system when it added a package named "incremental" to the `setup_requires` list in setup.py. Because `setup_requires` is a feature provided by setuptools, packages listed there are installed by (the same machinery as) setuptools' `ez_install` command, which does *not* respect `pip.conf`, and so it tried to download the package from upstream PyPI and timed out banging its head against the firewall. As a workaround, we messed with our build-scripts to manually "pip install" incremental before installing Twisted, or anything that depends on it, so everything's fine again. The Python Packaging Authority already has plans for replacing `setup_requires` with something more Pip-friendly (see PEP 518), so over time this won't be an issue. For the short term, though, I hope there won't be too many more things added to `setup_requires` without announcement. Tim.
On Dec 29, 2016, at 7:20 PM, Tim Allen <screwtape@froup.com> wrote:
On Thu, Dec 29, 2016 at 05:27:44AM -0800, Glyph Lefkowitz wrote:
Increasingly, we are assuming a pip-like packaging toolchain for dealing with Twisted's dependencies, so getting familiar with this stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile. (It'll make your life easier in more ways than one.)
I wasn't going to mention it (because we found a workaround), but since the topic has come up...
My current employer has a fully pip-based packaging toolchain for all our internal Python apps, where we build wheels for our apps and all their dependencies, and then deploy them into a virtualenv in production. For reproducability reasons, we have a PyPI mirror inside the corporate firewall, and our wheel-building Docker image has a `pip.conf` configured to point at it.
Twisted 16.6 broke this system when it added a package named "incremental" to the `setup_requires` list in setup.py. Because `setup_requires` is a feature provided by setuptools, packages listed there are installed by (the same machinery as) setuptools' `ez_install` command, which does *not* respect `pip.conf`, and so it tried to download the package from upstream PyPI and timed out banging its head against the firewall.
As a workaround, we messed with our build-scripts to manually "pip install" incremental before installing Twisted, or anything that depends on it, so everything's fine again.
The Python Packaging Authority already has plans for replacing `setup_requires` with something more Pip-friendly (see PEP 518), so over time this won't be an issue. For the short term, though, I hope there won't be too many more things added to `setup_requires` without announcement.
FTR, you can configure setup_requires too, it just uses a different format. In any of the files listed at https://docs.python.org/3.6/install/index.html#location-and-names-of-config-... <https://docs.python.org/3.6/install/index.html#location-and-names-of-config-...> You can drop a [easy_install] Index_url = https://private.example.com/simple/ allow_hosts = private.example.com # not strictly required, but will error instead of timeout for non private.example.com I’m not 100% sure on the exact syntax/structure, but something like that should work. — Donald Stufft
On Dec 29, 2016, at 4:20 PM, Tim Allen <screwtape@froup.com> wrote:
On Thu, Dec 29, 2016 at 05:27:44AM -0800, Glyph Lefkowitz wrote:
Increasingly, we are assuming a pip-like packaging toolchain for dealing with Twisted's dependencies, so getting familiar with this stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile. (It'll make your life easier in more ways than one.)
I wasn't going to mention it (because we found a workaround), but since the topic has come up...
My current employer has a fully pip-based packaging toolchain for all our internal Python apps, where we build wheels for our apps and all their dependencies, and then deploy them into a virtualenv in production. For reproducability reasons, we have a PyPI mirror inside the corporate firewall, and our wheel-building Docker image has a `pip.conf` configured to point at it.
Twisted 16.6 broke this system when it added a package named "incremental" to the `setup_requires` list in setup.py. Because `setup_requires` is a feature provided by setuptools, packages listed there are installed by (the same machinery as) setuptools' `ez_install` command, which does *not* respect `pip.conf`, and so it tried to download the package from upstream PyPI and timed out banging its head against the firewall.
Thanks, this is very useful information. If you wanted to have this verified for the future, a network-isolated buildbot would be a good way to ensure we don't mess this up in other ways. Based on this thread it seems that multiple users have this concern, so it's certainly not specific to you, either. It would also force us to fix the dang documentation-generator test cases that keep failing when a certain someone forgets to update their SSL cert :-).
As a workaround, we messed with our build-scripts to manually "pip install" incremental before installing Twisted, or anything that depends on it, so everything's fine again.
The Python Packaging Authority already has plans for replacing `setup_requires` with something more Pip-friendly (see PEP 518), so over time this won't be an issue. For the short term, though, I hope there won't be too many more things added to `setup_requires` without announcement.
Incremental is a bit of a special case, since "version definition" is one of the few things that needs synchronization between setup-time and run-time. I was pretty nervous about adding `setup_requires`, but the value of getting our versioning logic outside of Twisted itself seemed worth the potential packaging fallout. So, while you can expect to see many more things showing up in install_requires unannounced, as we split out low-level portions of Twisted and adopt libraries like h11, I doubt that you'll see another setup_requires arrive, especially before PyPA gets together a better approach than the literal setup.py setup_requires. Given that you've specifically asked for notice, any project member submitting or reviewing a patch that adds one should give a shout here. -glyph
On 12/29/16 08:27, Glyph Lefkowitz wrote:
On Dec 28, 2016, at 15:20, John Santos <JOHN@egh.com <mailto:JOHN@egh.com>> wrote:
Seems to require a new Python module, Automat.
Good catch, that probably should have been announced at least a little :).
(Not mentioned in the NEWS file, INSTALL, etc.)
Despite the fact that it would have been good to mention this particular dependency because its adoption is a new architectural direction for Twisted, in /general/ we're actually planning to do less communication of dependencies in this way. Right now, we have redundant listing of dependencies in a few places, but in the future, setup.py (or more specifically, "packaging metadata") will be the one, authoritative place that such things are listed.
I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format.
Although you probably /should/ just use the wheel, the fact that there was /only/ a wheel was an oversight. I've uploaded an sdist and pushed a corresponding tag (after making sure that the source files all matched, of course).
It is probably an off topic but it would not hurt to add a more verbose description and a link to the sources on PyPI for the Automat package especially if it is so important for Twisted.
On Jan 3, 2017, at 7:58 AM, Terekhov, Mikhail <termim@gmail.com> wrote:
On 12/29/16 08:27, Glyph Lefkowitz wrote:
On Dec 28, 2016, at 15:20, John Santos <JOHN@egh.com <mailto:JOHN@egh.com>> wrote:
Seems to require a new Python module, Automat.
Good catch, that probably should have been announced at least a little :).
(Not mentioned in the NEWS file, INSTALL, etc.)
Despite the fact that it would have been good to mention this particular dependency because its adoption is a new architectural direction for Twisted, in general we're actually planning to do less communication of dependencies in this way. Right now, we have redundant listing of dependencies in a few places, but in the future, setup.py (or more specifically, "packaging metadata") will be the one, authoritative place that such things are listed.
I found Automat 0.3.0 (with Glyph's fingerprints on it) at PyPI, but it only exists in .whl format.
Although you probably should just use the wheel, the fact that there was only a wheel was an oversight. I've uploaded an sdist and pushed a corresponding tag (after making sure that the source files all matched, of course).
It is probably an off topic but it would not hurt to add a more verbose description and a link to the sources on PyPI for the Automat package especially if it is so important for Twisted.
Please feel free to submit a PR to automat to do this. There's lots of descriptive material here: <https://github.com/glyph/automat/blob/master/README.md <https://github.com/glyph/automat/blob/master/README.md>> which you could use to update the PyPI metadata part. -glyph
participants (8)
-
Amber "Hawkie" Brown -
Donald Stufft -
ex vito -
Glyph Lefkowitz -
Jean-Paul Calderone -
John Santos -
Terekhov, Mikhail -
Tim Allen