Hello,
The other day, we had a Scrapy user report an issue connecting to
https://www.skelbiu.lt/ with OpenSSL 1.1 [1]
To not mix scrapy's things with Twisted Web, I used this (adapted from
official docs):
#---------------
from __future__ import print_function
from twisted.internet import reactor
from twisted.web.client import Agent
from twisted.web.http_headers import Headers
agent = Agent(reactor)
d = agent.request(
'GET',
'https://www.skelbiu.lt/',
Headers({'User-Agent': ['Twisted Web Client Example']}),
None)
def cbResponse(ignored):
print('Response received')
d.addCallback(cbResponse)
def cbShutdown(ignored):
print(ignored)
reactor.stop()
d.addBoth(cbShutdown)
reactor.run()
#---------------
And I did get a Handshake failure too:
$ python twistedtest.py
[Failure instance: Traceback (failure with no frames): <class
'twisted.web._newclient.ResponseNeverReceived'>:
[<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines',
'ssl3_read_bytes', 'sslv3 alert handshake failure')]>]
]
It seems this happens (at least) with OpenSSL 1.1.0e (currently in Debian 9
sid [2])
It does not happen (for me) with OpenSSL 1.0.2g for example.
I dug into this this afternoon and narrowed it down to the use of
_defaultCurveName = u"prime256v1"
in twisted.internet._sslverify.py
I tried patching the current trunk with _defaultCurveName = u"secp384r1"
(the EC that ssllabs.com reports)
and it did work.
Looking at ClientHello messages for openssl 1.0.2 and 1.1 [4]:
with 1.1, only 1 Elliptic Curve is sent by Twisted Web Agent, secp256r1
openssl v1.1 client uses 4 by default: ecdh_x25519, secp256r1, secp521r1,
secp384r1
I was wondering what is the proper way to configure requested Elliptic
Curves.
I haven't seen any interface for this, contrary to ciphers with
acceptableCiphers.
Thank you for your input.
Best,
Paul.
[1] https://github.com/scrapy/scrapy/issues/2717
[2] https://packages.debian.org/fr/source/sid/openssl
[3]
https://github.com/twisted/twisted/blob/78679af87e349721a167f35bef239e192e9…
[4] https://github.com/scrapy/scrapy/issues/2717#issuecomment-297464034
On behalf of Twisted Matrix Laboratories, I am honored to announce the
release of Twisted 21.2.0!
There are two major announcements for this release:
- *Python 2.7 support has been dropped*. Twisted 21.2.0 supports Python
3.5.3 and higher only
- *This will be the last Twisted release to support Python 3.5*.
Twisted 21.2.0 brings the following:
- twist dns --pyzone now works on Python 3
- twisted.web.twcgi now works on Python 3
- all tests pass on all platforms on Python 3.8, including asyncio tests on
Windows
- tests pass on Python 3.9 (thanks to Michał Górny)
- type hints, as specified in PEP 484, are now used in functions throughout
the codebase
- mypy (static type checker) is now run over the codebase during CI to help
detect problems, and help improve the use of types
- code in is now formatted with Black (thanks to Tom Most)
- many CI and automation improvements (thanks to Adi Roiban and Thomas
Grainger)
- many more fixes (225 tickets closed!)
You can install by running this command:
python -m pip install Twisted==21.2.0
The full tarball is available at:
https://pypi.org/project/Twisted/21.2.0/#files
The full NEWS file with all changes is at:
https://github.com/twisted/twisted/blob/twisted-21.2.0/NEWS.rst
