Twisted 23.8.0 Pre-Release Announcement
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
On behalf of the Twisted contributors, I announce the release candidate of Twisted 23.8.0rc1 This is a release that adds support for Python 3.12 This is also the last release with support for Python 3.7 The release and NEWS file is available for review at https://github.com/twisted/twisted/pull/11916 Release candidate documentation is available at https://twisted--11916.org.readthedocs.build/en/11916/ Wheels for the release candidate are available on PyPI https://pypi.org/project/Twisted/23.8.0rc1/ python -m pip install Twisted==23.8.0rc1 Please test it and report any issues. If nothing comes up in one week, I will do the final release based on the latest release candidate. Many thanks to everyone who had a part in Twisted development, the supporters of the Twisted project, the developers, and all the people testing and building great things with Twisted! I would like to use this opportunity and thanks https://thinkst.com/ for the extraordinary continuous financial support since 2018. I would also like to thank the Python Software Foundation and Phyllis Dobbs for their help in reviving the Twisted fiscal host. Slava Ukraini! -- Adi Roiban
![](https://secure.gravatar.com/avatar/8285ea1765ae52a3ab79c767676a2f4e.jpg?s=120&d=mm&r=g)
There is no "setup.py" How do I install from sources? I can't use pip for several reasons: 1) Customer systems are not Internet-connected and can't use pypi to retrieve packages at install time. 2) Version control. I need to be sure the versions of all packages pulled in due to dependencies exactly match what I'm using on my test/development systems. 3) I need to have actual source code, not an opaque blob, for documentation and accountability. On 8/17/2023 5:12 PM, Adi Roiban wrote:
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Aug 28, 2023, at 2:41 PM, John Santos <john@egh.com> wrote:
There is no "setup.py" How do I install from sources?
Pip is the only supported installer. If we want to support other installation mechanisms, we need to add them to CI first.
I can't use pip for several reasons:
Luckily it's totally possible to use pip and satisfy all these requirements!
1) Customer systems are not Internet-connected and can't use pypi to retrieve packages at install time.
Pip has an option, `--no-index`, which is explicitly for this sort of non-internet-connected system. For what it's worth, `setup.py` can and will also use setuptools to reach out to the internet to download stuff at install time; mostly, build dependencies. It's running arbitrary code and sometimes that code will attempt to shell out to easy_install.
2) Version control. I need to be sure the versions of all packages pulled in due to dependencies exactly match what I'm using on my test/development systems.
A great deal of pip's infrastructure is dedicated to allowing for transitive pinning of dependencies. You may want to look into pip-compile, specifically with the --generate-hashes option: https://pypi.org/project/pip-tools/
3) I need to have actual source code, not an opaque blob, for documentation and accountability.
Twisted and all of its dependencies ship source distributions as well as pre-built wheels to PyPI because many users have this requirement, so you don't need to use the wheels. The basic process for your type of build pipeline is two commands, although you can get a lot fancier if you want. The first command, you run on your development systems to build a meta-package of all the source distributions that you use as input to your development process; this needs Internet access and given your requirements this is the step at which you'd unpack all the downloaded archives and have a look inside them to review changes to validate updates for security, or whatever other purposes you need the source code for: 1. pip download --no-binary :all: twisted hatchling flit_core wheel calver setuptools_scm hatch_vcs hatch-fancy-pypi-readme This command is annoyingly long because I am not sure how to tell `pip download` to include all build dependencies to create a hermetic environment, but this is the current transitive list of all build-deps for Twisted's dependency tree, as far as I can tell. I think once everybody has adopted PEP 517 this will be simpler. If you need twisted extras like [tls], [http2], etc, or you want to use this process for a different project, the process of discovering these extra build-time dependencies is basically just do this step, do the next step, look for any package names in the installation error if you get one, then repeat. Note that "--no-binary :all:" will tell Pip to download source distributions only, for *all* packages. No prebuilt wheels, no binary components. 2. pip install --find-links . --no-index twisted This will install Twisted entirely from the source distributions you just downloaded with the previous command, no communication with PyPI at all, no loading of blobs. This is, of course, much slower than using all the prebuilt stuff and caching infrastructure one would normally invoke, it takes 1-2 minutes to get through this full process as opposed to only a few seconds for a `pip install`, but it satisfies your requirements. I would suggest adding a few extra steps though, since install-time for customers is quite different from verification-time for developers. Rather than doing a flat install directly off the source distributions on customer systems, I'd build your own wheels from the source of dependencies you've validated in-house, which would be something like: # on your build system pip wheel --find-links . --no-index twisted cp *.whl /.../installer-media # on customer system, later pip install --no-index --find-links /.../installer-media/ twisted Hope this helps! -g
![](https://secure.gravatar.com/avatar/8285ea1765ae52a3ab79c767676a2f4e.jpg?s=120&d=mm&r=g)
There is no "setup.py" How do I install from sources? I can't use pip for several reasons: 1) Customer systems are not Internet-connected and can't use pypi to retrieve packages at install time. 2) Version control. I need to be sure the versions of all packages pulled in due to dependencies exactly match what I'm using on my test/development systems. 3) I need to have actual source code, not an opaque blob, for documentation and accountability. On 8/17/2023 5:12 PM, Adi Roiban wrote:
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Aug 28, 2023, at 2:41 PM, John Santos <john@egh.com> wrote:
There is no "setup.py" How do I install from sources?
Pip is the only supported installer. If we want to support other installation mechanisms, we need to add them to CI first.
I can't use pip for several reasons:
Luckily it's totally possible to use pip and satisfy all these requirements!
1) Customer systems are not Internet-connected and can't use pypi to retrieve packages at install time.
Pip has an option, `--no-index`, which is explicitly for this sort of non-internet-connected system. For what it's worth, `setup.py` can and will also use setuptools to reach out to the internet to download stuff at install time; mostly, build dependencies. It's running arbitrary code and sometimes that code will attempt to shell out to easy_install.
2) Version control. I need to be sure the versions of all packages pulled in due to dependencies exactly match what I'm using on my test/development systems.
A great deal of pip's infrastructure is dedicated to allowing for transitive pinning of dependencies. You may want to look into pip-compile, specifically with the --generate-hashes option: https://pypi.org/project/pip-tools/
3) I need to have actual source code, not an opaque blob, for documentation and accountability.
Twisted and all of its dependencies ship source distributions as well as pre-built wheels to PyPI because many users have this requirement, so you don't need to use the wheels. The basic process for your type of build pipeline is two commands, although you can get a lot fancier if you want. The first command, you run on your development systems to build a meta-package of all the source distributions that you use as input to your development process; this needs Internet access and given your requirements this is the step at which you'd unpack all the downloaded archives and have a look inside them to review changes to validate updates for security, or whatever other purposes you need the source code for: 1. pip download --no-binary :all: twisted hatchling flit_core wheel calver setuptools_scm hatch_vcs hatch-fancy-pypi-readme This command is annoyingly long because I am not sure how to tell `pip download` to include all build dependencies to create a hermetic environment, but this is the current transitive list of all build-deps for Twisted's dependency tree, as far as I can tell. I think once everybody has adopted PEP 517 this will be simpler. If you need twisted extras like [tls], [http2], etc, or you want to use this process for a different project, the process of discovering these extra build-time dependencies is basically just do this step, do the next step, look for any package names in the installation error if you get one, then repeat. Note that "--no-binary :all:" will tell Pip to download source distributions only, for *all* packages. No prebuilt wheels, no binary components. 2. pip install --find-links . --no-index twisted This will install Twisted entirely from the source distributions you just downloaded with the previous command, no communication with PyPI at all, no loading of blobs. This is, of course, much slower than using all the prebuilt stuff and caching infrastructure one would normally invoke, it takes 1-2 minutes to get through this full process as opposed to only a few seconds for a `pip install`, but it satisfies your requirements. I would suggest adding a few extra steps though, since install-time for customers is quite different from verification-time for developers. Rather than doing a flat install directly off the source distributions on customer systems, I'd build your own wheels from the source of dependencies you've validated in-house, which would be something like: # on your build system pip wheel --find-links . --no-index twisted cp *.whl /.../installer-media # on customer system, later pip install --no-index --find-links /.../installer-media/ twisted Hope this helps! -g
participants (3)
-
Adi Roiban
-
Glyph
-
John Santos