I would greatly benefit from some direction regarding workflow. In
particular, how do you guys deal with having an installed version of
twisted, and a version (versions?) with which to tinker?
Currently, I have twisted installed in the normal end user way, and I have
a clone of my fork of the git repo. I mess around in the git repo clone,
but I don't know how to e.g. run the tests or run applications using the
git clone.
glyph explained this to me on irc a while back. Are the instructions from
that recorded somehwere?
Why am I asking? Because I want to do reviews and I'm not doing it because
I don't know how to set up a sane environment.
--
Daniel Sank
Not sure if this is the right mailing list for this....
epoll should *not* be used on 2.4 kernels...
I have 2.4 kernel running on a 32-bit x86 machine and another 2.4
kernel running on 64-bit x64 and I'm having an issue with twisted (via
buildbot).
I can import epoll from select on the 64-bit machine (it *should* fail
but doesn't) and this later causes twisted to fail...
>>> from select import epoll
>>> epoll(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 38] Function not implemented
On the 32-bit machine, however, importing epoll is properly failing...
>>> from select import epoll
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name epoll
>>>
twisted/internet/default.py is hitting this same problem. It's
incorrectly picking epoll when it shouldn’t because import epoll is
working when it shouldn't.
I 'fixed' the problem on my machine by adding a check method in
epollreactor.py which does this...
def check():
try:
epoll(1)
except IOError:
return False
return True
and I import and call the check method in default.py.
An alternate fix can be...
in twisted/internet/default.py
try:
if platform.isLinux():
try:
from select import epoll
epoll(1)
from twisted.internet.epollreactor import install
except (IOError, ImportError):
from twisted.internet.pollreactor import install
Any insight would be good. I had to compile my own Python 2.7.8 on
these old computers...maybe python is incorrectly detecting epoll in
the build process?
Thanks in advance,
Reuben
The porting guide says
No byte paths in sys.path.
I am not sure what this means, I would assume that
file paths should always be native strings. Or does it mean
that sys.path must only contain ascii bytes?
doc for FilePath says
On both Python 2 and Python 3, paths can only be bytes.
and svn commit 35410 by itamarst
changed the doc for some path functions in python/filepath.py
from str to bytes
but not all of them:
fgrep 'type path:' filepath.py
@type path: L{str}
@type path: L{str}
@type path: L{bytes}
@type path: L{bytes}
@type path: L{bytes}
I stumbled upon this while trying to find out how much work it might be
to make bin/trial run with python3
admin/run-python3-tests already passes for all twisted.spread related
tests but I still need to clean up a lot.
after adding an assert to FilePath.__init__, python3 bin/trial ... gives
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 601, in run
config.parseOptions()
File "/home/wr/ssdsrc/Twisted/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 472, in postOptions
_BasicOptions.postOptions(self)
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 382, in postOptions
self['reporter'] = self._loadReporterByName(self['reporter'])
File "/home/wr/ssdsrc/Twisted/twisted/scripts/trial.py", line 369, in _loadReporterByName
for p in plugin.getPlugins(itrial.IReporter):
File "/home/wr/ssdsrc/Twisted/twisted/plugin.py", line 209, in getPlugins
allDropins = getCache(package)
File "/home/wr/ssdsrc/Twisted/twisted/plugin.py", line 134, in getCache
mod = getModule(module.__name__)
File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 781, in getModule
return theSystemPath[moduleName]
File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 702, in __getitem__
self._findEntryPathString(moduleObject)),
File "/home/wr/ssdsrc/Twisted/twisted/python/modules.py", line 627, in _findEntryPathString
if _isPackagePath(FilePath(topPackageObj.__file__)):
File "/home/wr/ssdsrc/Twisted/twisted/python/filepath.py", line 664, in __init__
assert isinstance(path, bytes), 'path must be bytes: %r' % (path,)
AssertionError: path must be bytes: '/home/wr/ssdsrc/Twisted/twisted/__init__.py'
--
Wolfgang
Hi Twistosophists,
Some of you doing reviews (you ARE all doing code reviews, right!?!?) might have noticed that some of the boxes on the left side of of the supported builds page were dark grey, meaning that they were offline.
I spent some time over the last couple of days doing some administrative housekeeping and rebooting computers, and all of our supported builders are now back online! This makes me feel much better about the possibility of doing a release again someday :-).
Speaking of colors of boxes on that page, though, we should probably try to turn some of them from red to green...
-glyph
P.S.: Thanks to Rackspace for providing both (A) the cloud resources we're running many these tests on, and (B) my time to do this sort of maintenance.
doc says:
Instance: s is an instance of UserString.UserString, with a __dict__
{'data': 'hello'}::
[\"UserString.UserString\", ['dictionary', ['data', 'hello']]]
There is no test for instances. To be precise, no test ever enters
jelly._unjelly_instance()
I also cannot find any place in jelly.py where an Instance would
be jellied. There is
instance_atom = 'instance'
but instance_atom is not used anywhere.
Is this outdated code? If not, could you please give me a code
example using this?
--
Wolfgang
goal: Give knowitall.py awareness of wordsmith.py addition.
to run
./library.sh # opens 7999 for remote_dictionary and remote_define methods
then both:
python knowitall.py
python wordsmith.py
library.sh due to main/jelly issues
This does not seem to be supported by Python yet.
Should that be enabled at all?
If one process with PY3 sends such identifiers to
a separate process with PY2, that will fail. I am not
sure if that would be a problem, whoever uses this must
make sure PY3 is used everywhere.
If this should be forbidden, I will add a test to
test_pb for this. And of course somebody should document that
somewhere. There more PEP3131 is used, the more users will
fall into this trap.
If this should be enabled (which I think is not difficult,
at least for pb):
At least the patch below will be needed (only for PY3),
maybe it is already sufficient. Given that nativeString
and networkString are always used (done that for pb).
networkString may then return bytes with the high bit set
But since networkString is called in many places I want to ask and
make sure that it may really be changed this way.
https://twistedmatrix.com/documents/14.0.0/core/specifications/banana.html
does not speak against it, so I wonder why networkString has that limitation
to 7bit.
concrete banana-encoded example, from modified test_pb: (the method name is getSimpleä)
test_pb still passes with patched nativeString/networkString (but I
only have one test for this so far, test_refcount).
b'\x07\x80\x07\x82message\x01\x81\x03\x82foo\x0b\x82getSimple\xc3\xa4\x01\x81\x01\x80\x05\x82tuple\x01\x80\n\x82dictionary'
diff --git twisted/python/compat.py twisted/python/compat.py
index 6f76c39..6919cf6 100644
--- twisted/python/compat.py
+++ twisted/python/compat.py
@@ -348,10 +348,9 @@ def nativeString(s):
raise TypeError("%r is neither bytes nor unicode" % s)
if _PY3:
if isinstance(s, bytes):
- return s.decode("ascii")
+ return s.decode("utf-8")
else:
- # Ensure we're limited to ASCII subset:
- s.encode("ascii")
+ return s
else:
if isinstance(s, unicode):
return s.encode("ascii")
@@ -428,7 +427,7 @@ if _PY3:
def networkString(s):
if not isinstance(s, unicode):
raise TypeError("Can only convert text to bytes on Python 3, I got %r" % (s,))
- return s.encode('ascii')
+ return s.encode('utf-8')
def networkChar(integer):
"""
--
Wolfgang
Hi,
While using spawnProcess on Linux I found out that when an invalid
executable is called there is a corner case in which a zombie process
is left until main process exists and can not be closed.
I wrote a test for this but I was not able to reproduce this error in
isolation, event if I run the test for 10000 times. reapProcess will
always succeed from the first call.
For the production code I can always reproduce the problem.
Inspecting the execution thread I found out that all pipes are closed
but spawned process is not closed yet. Due to this
Process.maybeCallProcessEnded() will call self.reapProcess().
In my case, os.waitpid(pid, os.WNOHANG) return 0, and
self.reapProcess() will just ignore this case.
Process handlers is still registered in reapProcessHandlers but
reapAllProcesses is no longer called.
If a add a reactor.callLater(self.reapProcess) the next call to
os.waitpid will succeed and the forked process is closed.
------------
What am I doing wrong? Why reapAllProcesses is no longer called, even
if reapProcessHandlers is not empty?
My process protocol code is here
https://gist.github.com/adiroiban/bac493f00ce5e94738ce but from what I
see, this should happen for any protocol.
Many thanks,
--
Adi Roiban
Hello list,
I need to implement a graceful shutdown procedure for a twistd application.
The application is made up of two services: an internet.TCPClient and
an internet.TCPServer.
They're glued together with a MultiService instance, which is in turn set
to have 'application' as parent.
The server and the client work together, making a proxy (SMTP server and
AMQP client).
My goal is the following:
- intercept a SIGTERM signal
- 'block' on the server side: since it's SMTP I get this by setting a
variable that makes the server return tempfails (4xx) for new messages,
while keeping current sessions active
- wait until current requests are satisfied (I keep a dictionary of current
pending messages)
- shut the whole thing down
What is the best solution for this use case? It's not really clear to me
how to catch SIGTERM and handle pending requests *before* the underlying
services start to shutdown (i.e. even addSystemEventTrigger('before',
'shutdown', callable) is called too late for my needs).
Thank you very much for your help!
Fabio
