Twisted July 2017

twisted@python.org

18 participants
22 discussions

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing

by Glyph Lefkowitz

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

2 months, 3 weeks

[Twisted-Python] Real Viagra Professional - for real sex pros

by Tamika

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html>

2 years, 6 months

[Twisted-Python] Nevow 0.14.3rc1 released

by Tristan Seligmann

Hi all, I've uploaded a prerelease of Nevow 0.14.3 to PyPI[1]. This contains a fix for some Athena client-side timeout behaviour (or more specifically, the lack of one) and removes use of FilePath.getmtime. If you are using Athena, please a) let me know! and b) give this prerelease a try to see if anything breaks. As the de facto Nevow release manager, and one of its last users, I'd like to avoid going overboard on release engineering if it isn't necessary, but I'd also like to avoid making life worse for any other remaining users, so it would be useful to hear from any of you out there still using Nevow. [1] https://pypi.python.org/pypi/Nevow/0.14.3rc1

4 years, 2 months

[Twisted-Python] Using Twisted with pytest fixtures

by Hamza Sheikh

I use pytest[0] to write integration test automation. If the tests are stripped down to their core, they are basically using different network connections and protocols to drive multiple applications. Tests use the following Python libraries: * paramiko[2] * docker[3] * requests[4] * pychef[5] * pymongo[6] * python-qpid-proton[7] * websocket-client[8] I use pytest fixtures[1] heavily. Each of the aforementioned libraries is used such that each connection (say SSH) is provided to tests through that fixture. This makes writing tests quite simple as a single object can be used to maintain a network connection that can be reused across multiple tests in the same session. In case the connection needs to be re-established, the class has methods to hide that detail from the tests and things just work. Sample code is presented below: # In conftest.py from path_to_lib.ssh import SSH @pytest.fixture(scope="session") def ssh_fixture(): return SSH( host=hostname_, port=22, username=username_, pem=path_to_pem) # In path_to_lib/ssh.py import paramiko class SSH(object): def __init__(self, host, port, username, pem): self.host = host self.port = port self.username = username self.pem = pem self.client = paramiko.SSHClient() self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) self.transport = None def _connect(self): self.client.connect( hostname=self.host, username=self.username, key_filename=self.pem, port=self.port) def exec_cmd(self, cmd): try: _, stdout, stderr = self.client.exec_command( cmd, get_pty=True) except: self._connect() _, stdout, stderr = self.client.exec_command( cmd, get_pty=True) return stdout, stderr # In path_to_tests/test_foo.py def test_foo(ssh_fixture): cmd = "hostname" stdo, stde = ssh_fixture.exec_cmd(cmd) assert stdo # In path_to_tests/test_blah.py def test_blah(ssh_fixture): cmd = "ls /" stdo, stde = ssh_fixture.exec_cmd(cmd) assert stdo The above sample code is essentially how all other libraries (say for Mongo) are used in my tests. Each type of protocol is presented to tests through a fixture and tests only need to call certain methods without having to know the gory details behind them. This method provides very readable code. It also makes writing tests much easier. In addition to using these libraries sometimes I need to use the socket library to talk to proprietary services using TCP and/or UDP that don't have open source libraries available. The problem is that I have a *lot* of these integration tests that can run for a long time (anywhere from 2 hours to 12 hours based on the test set selected) (I need to test a *lot* of services and components). This is causing some issues when I get my hands dirty with the socket library. Instead of writing a custom and badly implemented Twisted alternative I thought I could use Twisted to replace a bunch of these libraries. Of course, I can't replace all of them at once today but I'd much rather start on the path. As I started getting familiar with the official and community docs I realized that most of them deal with writing servers (and sometimes clients). I can't find any docs on how people in my situation use Twisted. >From what I understand -- forgive my beginner-level grasp of the subject matter -- once the reactor starts no more listenTCP() or connectTCP() (for example) can be added. And until the reactor stops code execution does not progress beyond ``reactor.run()``. For example, in the code below, _Done_ is not printed until there's a ``reactor.stop()`` somewhere above it. blah ... reactor.run() print("Done") My questions for the community: * How can I use Twisted that fits the way my tests are written? * If I have to rewrite my tests to fit the Twisted model, do I start the reactor every time for each test and stop it at the end? * Are there any known issues with start/stop reactor multiple times in a single, long-running process? * Are there any good examples of using Twisted in integration tests that I have failed to find? [0] https://docs.pytest.org/en/latest/ [1] https://docs.pytest.org/en/latest/fixture.html [2] https://pypi.org/project/paramiko/ [3] https://pypi.org/project/docker/ [4] https://pypi.org/project/requests/ [5] https://pypi.org/project/PyChef/ [6] https://pypi.org/project/pymongo/ [7] https://pypi.org/project/python-qpid-proton/ [8] https://pypi.org/project/websocket-client/ Thanks in advance. Hamza Sheikh Twitter: @aikchar

4 years, 2 months

[Twisted-Python] Weekly Bug Summary

by trac＠twistedmatrix.com

4 years, 3 months

[Twisted-Python] quality of non-member reviews

by Glyph

Hello Twisted Project Members (i.e.: committers, people with repo:write access), Whenever you get a code review from a non-member, it's important to always[1] ensure that they remove the 'review' keyword. This is not really important in and of itself, but it is very important for other reasons, which I will attempt to explain, since I think I may not have properly laid this out sufficient detail before. The goal of allowing non-project-members (i.e.: external contributors) to review changes in Twisted is to provide a pipeline for folks to become developers of the project; to allow people to gain experience with reviewing, so they can proceed to being full reviewers, without requiring that they already be project members before they can practice. In other words, it is a process to train reviewers so that we can have higher code-review throughput in general in the future, not a shortcut to get changes reviewed more quickly. It is therefore the responsibility of the developer accepting the code review to ensure that the reviewer is following directions and has given sufficient attention to the review, and to provide feedback on those reviews to ensure that the new contributors are learning how to look for the things that help us maintain Twisted's quality. Sometimes, the only code review you need is a "thumbs up, looks good", because everything really is fine, the fix is simple, and there's nothing much the reviewer could say. Ideally, all changes are small, and obvious, and reviewers can just give that review regularly. However, such a review is always suspicious, because it's very easy for a reviewer, especially an external reviewer with little understanding of the project, to just rubber-stamp a fix they want to see merged quickly without giving it any real analysis. As such, it's very important that when accepting a very short review that you use whatever clues you have at your disposal that the reviewer has actually reviewed the code in some detail. One potential proxy for this is that our process requires some small peculiarities, such as attaching and removing the 'review' keyword in Trac[2]. If a contributor has not even read the instructions carefully enough to know that they need to remove this keyword, it is virtually impossible for them to have dedicated the attention and care required by https://twistedmatrix.com/trac/wiki/ReviewProcess#Howtobeagoodreviewer <https://twistedmatrix.com/trac/wiki/ReviewProcess#Howtobeagoodreviewer> to do a full review and consider everything that cannot be automatically verified, such as impact to Twisted's public interface, the legibility of documentation, duplication of features, et cetera. If the contributor has demonstrated their attention to detail in some other way, by giving a thorough review and writing some analysis of the change's implications (even if it's not actionable, just something like "I read over the narrative documentation to ensure it didn't need any changes and it looks like we're good") then it might be acceptable to gently point out the small process mistake of not updating trac. But if the change is just a Github comment saying "LGTM" with nothing else associated, it is important not to accept the review and to explain that the reviewer should re-read the directions explaining how to give a review. As you might have guessed, I have seen "LGTM" changes landing on trunk recently whose reviews looked somewhat suspicious to me. I, and a few other long-time Twisted developers, have complained about this before, but in re-reading those complaints in preparation for writing this message, it occurred to me that I may not have spelled out exactly what I was concerned with and why, so, in the spirit if blameless post-mortems, I don't want to call out anyone specifically[3]. Pascal's apology applies; I haven't had much time to edit. Thanks for reading, -glyph Endnotes: [1]: Iron-clad process rigidity is not the goal here. Sometimes there are legitimate reasons to need to move things along a little faster than a full thorough review requires. For example, if there's a pressing security issue, or all the CI infrastructure is failing due to a bug in one of our dependencies. There are also cases where the important thing that needs to be done in a review is a highly specialized analysis, like; for example, if we need a GPS expert to verify that some change to twisted.positioning is correct, and beyond the domain knowledge the code change is otherwise trivial. In these cases it might be fine to accept a "LGTM", if the question posed in the review is highly specific, but in that case, it's important to leave a comment explaining what exactly has transpired: "I'm removing the review keyword myself because I needed a domain expert to review this, and XXX YYY was helpful enough to have a look but is not familiar with our process." or whatever the case may be. In general, putting more thorough comments on tickets is always a good thing. However clear that you as a Twisted project insider think a thing is, chances are it is about 1/4 that clear to anyone trying to understand project direction from the outside. I'm certainly guilty of under-documenting myself :-). [2]: we are not preserving Trac purely as a shibboleth to check on people installing the directions; software development is hard enough without extra roadblocks thrown in the way. There are just too many tools connected to the 'review' keyword right now that are hard to connect to Github automatically. Whatever our process, however, it's important to verify that reviewers are following it carefully. [3]: I realize that there are too many bits of identifying information here to be meaningfully anonymous, so some folks will immediately know the specific ticket that precipitated this, but don't be a jerk and dig up someone's name from the trac timeline because you can; the goal here is not (and should not) be to throw anyone under the bus, but to improve communication around project policy.

4 years, 3 months

[Twisted-Python] Weekly Bug Summary

by trac＠twistedmatrix.com

4 years, 3 months

[Twisted-Python] Incompatible changes to twistd 16.3.2 to 16.4.0

by Thomas Westfeld

Dear all, I am experiencing an issue with my twistd plugin. It is in a project_plugin.py file in the twisted/plugins directory of my project. Calling "twistd --help" works fine up to twistd version 16.3.2 but in 16.4.0 the plugin is not recognized anymore. This is how it looks like: from zope.interface import implementer from twisted.application.service import IServiceMaker from twisted.application import internet from twisted.plugin import IPlugin from twisted.python import usage from myfile import MyFactory class Options(usage.Options): optParameters = [["port", "p", 8000, "The port number to listen on."], ["my-hostname", "l", "", "the hostname of other server"], ["my-port", "i", 9000,"the port of the other server"]] @implementer(IServiceMaker, IPlugin) class MyServiceMaker(object): tapname = "myserver" description = "" options = Options def makeService(self, options): return internet.TCPServer(int(options["port"]), MyFactory(options["my-hostname"], options["my-port"])) serviceMaker = MyServiceMaker() Any help is appreciated. Thomas

4 years, 3 months

[Twisted-Python] Weekly Bug Summary

by trac＠twistedmatrix.com

4 years, 3 months

[Twisted-Python] Failure of twisted.test.test_plugin.DeveloperSetupTests.test_freshPyReplacesStalePyc under pypy2

by Daniel Sutcliffe

I'm getting this report under pypy2 5.8.0 running Twisted trunk from github: =============================================================================== [FAIL] Traceback (most recent call last): File "/opt/twisted/src/twisted/test/test_plugin.py", line 566, in test_freshPyReplacesStalePyc self.assertIn('one', self.getAllPlugins()) File "/opt/twisted/src/twisted/trial/_synctest.py", line 492, in assertIn % (containee, container)) twisted.trial.unittest.FailTest: 'one' not in ['dev', 'app'] twisted.test.test_plugin.DeveloperSetupTests.test_freshPyReplacesStalePyc =============================================================================== [ERROR] Traceback (most recent call last): File "/opt/twisted/src/twisted/plugin.py", line 171, in getCache provider = pluginModule.load() File "/opt/twisted/src/twisted/python/modules.py", line 392, in load return self.pathEntry.pythonPath.moduleLoader(self.name) File "/opt/twisted/src/twisted/python/reflect.py", line 312, in namedAny obj = getattr(obj, n) exceptions.AttributeError: 'module' object has no attribute 'stale' twisted.test.test_plugin.DeveloperSetupTests.test_freshPyReplacesStalePyc ------------------------------------------------------------------------------- I'm not sure yet if it works under pypy3. My initial tracings seem to suggest this is somehow due to __import__ under pypy not finding the stale.pyc where no stale.py exists ... but I have to admit that at this point if this is actually expected, all I know is the test passes under python 2.7.13 and if I comment out the mypath.remove() on line 562... Willing to do more work to get to the bottom of this, just thought I'd put this out there to see if I could get any clues from the more experienced before I dive in. There seem to be some old tickets that could vaguely relate... Am I correct in the fact that Twisted doesn't have Travis-CI builds on PyPy? Maybe there's a reason for this I'm missing... Thanks in advance for any info or clues, /dan -- Daniel Sutcliffe <dansut(a)gmail.com>

4 years, 3 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

Twisted July 2017