Please indulge me -- my reasons are at the end.
Twisted as a House for Sale
You double check the listing printout to make sure you're not lost.
On the drive in, you didn't see many other cars on the road or people
out on the street -- the neighborhood seems kind of deserted.
"Nope, it's the right place," you say as you turn the key toward you.
The engine stops, and the resulting silence is immediate. Ronnie, the
agent who's going to show you the place, said he'll meet you inside.
You first notice the yard. The grass is a little yellowed, except
where it grows abundantly at the base of the waist-high, once-white
picket fence. There are a bunch of signs on the fence and in the
yard. Many of the signs have, "Attn Construction Workers" written in
big red letters at the top. You walk toward one to see what it says,
and then... in the middle of the front yard, you notice a freestanding
door. It looks new. Fresh paint. Shiny knob and knocker. Standing
at an angle, right in the middle of the lawn.
Forgetting the signs for a minute you walk to the fence, open the gate
and make your way up the walkway toward the freestanding door.
Standing in front of the door you read, etched into the knocker,
"Weird," you say again. It almost feels like a tombstone.
You turn back toward the house. There are lots of windows (way more
than at that other house on Amigofeed Drive). And you notice for the
first time that there are three front doors.
"Umm...", you're beginning to doubt that this is really the house you want.
You step onto the porch and examine the doors. From left to right,
they are labeled "reactor.listen," "TCP4ServerEndpoint" and
"serverFromString." You reach for TCP4ServerEndpoint, but second
guess yourself and instead open the "reactor.listen" door. You're
standing in a small entryway. There's ample light coming from a
fluorescent fixture, though the light seems a bit sterile. Kind of
like an old hospital.
"Oh man," you smile. You just noticed the not-quite-lime-green shag
carpet. Pretty sweet.
It still feels eriely quiet. From how Ronnie described this place
over the phone, you were expecting a little more. This dismal room
just seems so stagnant. You glance at your watch, wondering when
Ronnie's gonna show.
"I guess I can poke around a little while I wait."
You build up a nice static charge shuffling toward the door at the
other side of the entry way. Zap! You get a short blue arc as you
grab the doorknob and turn. You glance up and notice a small label on
the door as you push it open. The label says "reactor.run()."
A huge wave of sound washes over you, startling you! There are people
everywhere. Some are running around, some are sitting and talking,
some are arguing loudly. There's people on phones and on computers.
It's a madhouse of activity! This room is huge! It's cavernous. It
feels like Grand Central. The ceiling must be upwards of 30 feet
high! The room is full of sunlight, streaming in through multiple
skylights and through the wall opposite you (perhaps 60 feet away)
which is made entirely of glass. The glass wall looks out to a
beautiful garden, full of trees and flowers.
"Hi, I'm Ronnie!"
You swing around, surprised.
"This place is amazing!" you exclaim.
"Sure is. Let me show you around!"
"But why is the front..." you start.
"This room is the event loop," he cuts you off. "Everything goes
through here. Well, not everything. Deferreds don't, and some of the
other utilities" he continues, grabbing your arm and pulling you
through the room.
"De--whats?" you stammer.
He stops walking and turns toward you. "Deferreds. You know, like:
'I'm going to hit you'." He pauses.
"You're going to wh... OWW!" He slaps you hard across the face.
"Deferreds." He grabs your hand and starts walking briskly through
the room again. You rub your cheek wondering what just happened.
"Trap it next time. Let me show you the library!"
"This place has a library? Cool. I hope it has one of those sliding
ladders," you think to yourself, still nursing your red cheek.
Ronnie takes you through a door at the side of the room. It smells
like a library, but it seems awfully small. For the asking price,
though, you can't complain.
"What do you want to know about?" he asks.
"What do you want to learn? Fencing, cooking, horitculture, aviation,
rare-earth metal mining? What do you want to know?" he asks again,
looking right at you and waiting.
"Alright, this way." He grabs your arm and yanks you through a door
you hadn't realized was there. You find yourself in a room no bigger
than a closet, smashed right up against Ronnie.
"Here you go." He shoves a paper into your hands.
"Ye Olde Guide of Squirrels Cooking" reads the title.
"How old is thi..." you begin to ask.
Ronnie grabs the paper back, examining it with furrowed brow.
"Bah!" He exclaims, "those kids keep messing with the index. There's
not much we can do about that. Let's go to the most recent version."
He leads you through a door to the right.
You're in another closet. He shoves another paper into your hands.
"How to Cook a Squirrel" reads the title. You start perusing it,
forgetting why, exactly, you are reading it in the first place. And
it's a little uncomfortable being in these tight quarters with Ronnie.
He's awkwardly silent and motionless standing there, smiling, staring
at you while you read. After reading the first few paragraphs you
ask, "Is this the best thing you have on cooking? I mean, squirrels
are nice, but..."
"We've got other stuff." He pushes the opposite wall, which, it turns
out was a door, and you find yourself in yet another closet. He
shoves another paper into your hands.
This one is "Proper Care of Cutlery."
"Umm... so, anyway, this library is great. But what else can you
show me in this house?"
"Oh, there's lots of things. But you should really check out the
library some more. It's full of great stuff. And make sure you check
all the sources."
He pushes through to the door you came in, then through another door,
then another, and... you are not back in the first room, but another
closet. Oblivious, Ronnie, pushes through another door leading to a
closet, then another, then another, then another. You're not sure
you'll ever get back to where you were or that you'd have the
slightest chance of finding these rooms again.
"I think if I go with this house, I'll need to make myself a map of
this library," you think, as you follow Ronnie from closet to closet.
Finally, he pushes through a door and you walk back into the first
room of the library.
"Let me show you some more of the house," Ronnie smiles as you head
toward the event loop room.
Ronnie leads you on a tour of the home. It's rather impressive. And
expansive. While in the basement, which is well lit and surprisingly
inviting, he points out the reinforced concrete foundation built
around massive steel beams which have been driven eighty feet into the
ground. The foundation is incredibly strong. You didn't realize it
from outside, but this house was designed to stand for years -- it's
In the bathrooms and kitchen, Ronnie proudly shows you the plumbing.
It's solid stuff. The craftsmanship is excellent, and it looks like
everything is built to last.
"But, just wait," he says, with a twinkle in his eye. "We're getting
some new plumbing installed soon."
"Oh that's great. Replacing plumbing is one of the best ways to..."
"I didn't say 'replacing.' We're getting new tubes, but we'll still
have the old stuff, too."
During your tour, you go up and down stairs (it feels like some of the
staircases have one too many steps). And oddly, there are a lot of
rooms with two or three doors leading into them from the same hallway.
You notice small manhole covers in many of the hallways. After
passing the fourth or fifth one, you ask Ronnie what they are.
"Oh, those are manhole covers." He casually responds, not stopping.
Still confused, you try again: "What are they for?"
"Anything you want."
Thanks, Ronnie. "For example?"
Not quite sure what's so confusing, Ronnie explains "Oh, you know.
You can fix plumbing, add new walls or rooms, elevate the roof,
activate the alarm, lock all the doors, release the badgers. Just...
whatever you want."
Badgers? Intentionally sidestepping the badgers, you ask "Don't
people get upset when you turn off their water to fix the plumbing?"
"Why would we need to turn off the water just to fix the plumbing?"
Ronnie asks, puzzled.
You decide not to press it.
Eventually you end up back in the event loop room by the reactor.run()
door. You've got another appointment, and it seems like you've seen
most of the house.
"Thanks for the tour, Ronnie," you say, "I'm really impressed with the
quality of work in this home. There's a lot of hallways and doors to
remember, but it's very nice." You hold out your hand to shake.
"Are you going already?" Ronnie asks, sadly, "I was just about to show
you the sails!"
"Yeah, we're going to put out to sea any minute now."
"The house comes with a boat?" you ask incredulously.
"No, that's ridiculous. The house *is* a boat," he says, like it's
the most normal thing in the world.
"The house is a boat?"
"The house is a boat."
"I'm on a boat?"
"Yes." He says, a little hesistantingly. Then, a little worried and
not sure what to do with his hands, "Is it... uh... strange that the
house is a boat?"
"Um... well... yes. I've never looked at a house that was also a boat."
Ronnie seems disappointed. "Really? That's too bad. It's really
nice having a house that's also a boat." Then, with the smile coming
back into his face, he says "but you don't have to use it as a boat!
And it being a boat doesn't get in the way of it being a house!"
"Err... great..." you begin, but he interrupts in his excitement.
"In fact," he goes on excitedly, "you'll probably be happy with just
the air travel."
"Air... travel?" you ask, reaching for the door.
"Of course!" he shouts in complete disbelief.
"You've got to be kidding me," you say.
"YOU'VE got to be kidding ME!" he says, in shock. "What are those
other people trying to sell you out there!? Don't tell me the other
houses just sit on the ground! They don't fly?" Ronnie slowly nods
his head as if to coax the answer out of you. But you don't mirror
"I don't believe it!" he continues. "Next you're going to tell me
that when they make an addition to their flightless, floatless houses
that they don't even test to make sure that the addition will stand."
He's serious. The house flies. You're dumbfounded. This house, with
the unkempt front yard and shag carpet, located in a seemingly
desolate part of town can fly. Why isn't anyone talking about this!?
Bzeerrpp. Your phone is ringing. You're gonna be late for your appointment.
"I really need to go, Ronnie. But I'll definitely be back. Thanks
for everything. This house is amazing."
You turn toward the reactor.run() door and start turning the doorknob.
"STOP!" A hundred voices yell. Panic-stricken, you jump back from the
door and whip around, breathing heavily from the surprise. Everyone
in the room including Ronnie is frozen, staring at you.
"That was close!" he says, "you only go through that door once." He
seems relieved you didn't go through.
"Here, come this way." He shows you to a door labeled
"reactor.stop()" and sends you on your way.
I wrote this because it's frustrating to see people make new, shiny
toys out to be amazing and revolutionary when they're not built on a
solid foundation, are not new, and are not even as good as existing
stuff (Twisted). Twisted can fly!
I would love for more people to step inside the house and see what it
can do, instead of judging it based on incorrect facts or uninformed
opinions. Or, more to my point, judging it based on the website
(geared toward people working on Twisted), documentation, examples, or
multitude of ways to do the same thing because of backward
I want to mow the lawn, so to speak. I want to plant flowers and make
the One Front Door more prominent. I'm considering making a site
dedicated to people learning Twisted. It would be cool if it there
were things showcasing running Twisted services (websockets + ssh +
irc or something).
Is there interest in this?
Also, what do you think of the story? :)
(Sorry for the long email.)
This doesn't prove anything, but I think the first plot at
http://jakevdp.github.io/blog/2012/09/20/why-python-is-the-last/ is quite
Semi-related: I made a tiny optimization to jQuery a couple of months ago
as a completely unknown first-time contributor. The time from doing the git
clone to having the pull request merged into jQuery master was about an
hour, which included signing their contributor agreement.
I recently created a small automated, remote (via web) controlled pet
feeder using a Raspberry Pi single-board computer. The software is all in
Python, and uses the simple HTTP server that is part of the Python
libraries. I discovered Twisted about 2/3 of the way through the project,
and now that I am done I would like to port it to use Twisted. After
reading much of the documentation the Twisted web site, some aspects of
this are fairly obvious to me, while others are not. I could use some
advice on how best to design my app to fit in with the Twisted framework.
Here are the major components of the application:
1) A web site that hosts a combination of static content and some REST
enable/disable the webcam, dispense a treat, etc. The static content
captured by #3 below.
2) A thread that runs a loop which manages the hardware. It does this with
a simple state machine composed of a base class and subclasses for all the
states that the hardware can be in (starting, idle, displaying status on
the lcd, dispensing a treat, recovering from treat dispense cycle, shutting
down). After initializing the hardware and the initial state, the thread
loops continuously calling a "timeTick" event to the current state. WIthin
the states, the code looks at various hardware status (like whether a
button is pressed) and decides to take action or to trigger transition to
3) A thread that runs continuously capturing images from a webcam. This
thread captures low resolution images continuously, and compares sucessive
frames to see if there is significant number of different pixels (motion
detect). If so, it captures a higher resolution image, updates a symbolic
link to point to the most recent image captured, and deletes any excessive
files from previous captures. The images captured for the motion detect are
handled by executing a process and capturing the stdout. The higher res
images captured by executing a process that writes directly to a file.
Porting #1 is very straightforward after going through the tutorial on
Twisted web. However, I am not sure how to handle #2 and #3. I would
appreciate some suggestions from those who are familiar with Twisted.
I'm want to get the information of 25(smtp) port in the target ip ,so
i write the script smtp.py,and the mx.short.txt is the target ip list.You
can run the script like this :python smtp.py(the smtp.py and mx.short.txt
are in the same directory).Then I find this script is can't stop.
Can you help me solution this problem?It's had troubled me about one
just started using Twisted and don't know how to manage
communications in a monitoring application I'm building for Windows
I'm building an application where various client
machines will be receiving real-time updates on the status of multiple
servers (CPU/RAM usage, tasks running, ...). Clients should also be able
to send commands to server machines (Start/Stop task, Shut down,
What protocol should I use? I managed to get
something running from a simple Echo server sample, but I guess it's not
the best curse of action as I could end up having 10 machines
establishing a connection every few seconds to each of the 20 or so
servers we have.
Is there a sample of a similar setup I could follow?
I'm having several problems with tac2deb:
* First, it seems that the documentation is wrong. Like described in: https://twistedmatrix.com/documents/current/core/howto/tutorial/configurati… as any call to tap2deb as tap2deb --unsigned -m "Foo <foo(a)example.com>" --type=python finger.tac just ends with a /usr/bin/tap2deb: Wrong number of arguments.
* When using the -t option, it seems it works, but it always ends with this kind of error:
dh_installdirs: No compatibility level specified in debian/compat dh_installdirs: This package will soon FTBFS; time to fix it! # Add here commands to install the package into debian/tmp. cp tap.py debian/tmp/etc/ cp: cannot create regular file `debian/tmp/etc/': No such file or directory make: *** [install-stamp] Error 1 dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
I tried to upgrade twisted to latest version (13.1), but it did not fix the problems.
So, do we have a working example of using tap2deb with a recent twisted release (12-13)?
I have written a simple serial-to-serial logging proxy in order to discover and document the protocol used by two hardware devices. The goal is to incrementally replace one of the devices with a Twisted application, but right now I'm just trying to gather data.
A RaspberryPi seemed ideal for this given its cost and ability to run python; however, I seem to be running into some kind of timing or data corruption issue that only occurs on the Pi, and only when I am logging data. The application works well on my dev laptop (OSX), but falters on the Pi.
I can use socat to do raw data collection for this first stage (it works fine on the Pi as a logging proxy), but I was hoping to use Twisted with custom log observers to format the data in ways that might help illuminate the framing of the protocol. The idea was to have the protocol just log and proxy each byte and then I could experiment with a series of observers to identify the patterns (emphasize STX/ETX as possible delimiters, flag this sequence as an 'INIT' sequence, etc..)
[CON-A] <----> [SerialSpy (Twisted)] <----> [CON-B]
When CON-A is powered-on, it goes through an init sequence to establish communication with CON-B and if there are any issues, CON-A gives up and displays an error message.
When run from my dev laptop (OSX), the data is logged and proxied between the 2 devices without any problems.
When run on the Pi (Raspbian), back and forth comm is fine for the first 4-5 framed packets (STX...ETX+CKSUM), but then CON-B begins to NAK every packet after that and CON-A gives up.
The total amount of data transferred during a successful init sequence is < 1k:
CON-A: 537 bytes
CON-B: 290 bytes
A minimal Protocol subclass logs each byte received and writes to its peer.
class ProxyLoggingProtocol(protocol.Protocol, object):
peer = None
def __init__(self, name):
self.name = name
def dataReceived(self, data):
Log data received and proxy to peer
for b in data:
# Passthrough data to "other" transport
if self.peer is not None:
err = ValueError('Peer protocol not set (nowhere to proxy this data)')
... setup the protocols and serial transports:
conA = ProxyLoggingProtocol('CON-A-->')
conB = ProxyLoggingProtocol('<--CON-B')
conA.peer = conB
conB.peer = conA
SerialPort(conA,'/dev/ttyUSB0', reactor, ...)
SerialPort(conB, '/dev/ttyUSB1', reactor, ...)
It seems that doing anything in dataReceived beyond just proxying bytes from A to B is enough to cause CON-B to interpret received data as either bad or delayed to the point where it returns a NAK. Oddly, the first few framed packets are logged, proxied, and ACKed successfully.
This occurs if I log.startLogging() to a file or stdout. If I never call startLogging(), the data proxies without error. That is, CON-B replies with positive ACKs and everything works as expected.
I suppose the obvious answer may be that the Pi is underpowered for this task, but I am hoping to better understand why. I may be able to defer logging to a queue or find other workarounds, but I'd like to figure out why basic usage of logging could be causing this. Any thoughts as to what I could be looking at next would be appreciated.
I know that this has been asked before, but it's been a while and I'm
hoping for some good news. Is there a SQL ORM that works well with
Twisted and PostgreSQL? In particular I'm hoping to find something
that works with txpostgres as that's the library I prefer to access
the database with.
>> 1. That there is a consent on high quality ciphers: for example right
> now there are roughly two fractions who agree what is the lesser evil: RC4
> or AES-CBC.
> No, it is now clear that RC4 is the greater evil. The browsers have
> deployed defenses against the "BEAST" attack on CBC (the defense is "1/n-1
> record splitting"), and BEAST is an active attack which can only be used
> in some cases and which tends to leave evidence of the attempt. On the
> other hand, RC4 is apparently vulnerable to passive attacks, which are
> more serious.
> (If I'm wrong and there actually *is* a faction who still prefers RC4
> despite the recent results against it, I'd like to read about it!)
I’m not going to argue ciphers with you because you’re obviously right and I already wrote elsewhere that I’m going to full defer to your judgement here.
To explain where the above came from and eg. Qualys is still somewhat for RC4 as a fallback cipher: to the best of my knowledge, Apple’s desktop Safari browser ''still'' hasn’t activated record splitting in its latest version and is thus still vulnerable to BEAST (and doesn’t support TLS>1). But that’s probably a corner case enough to ignore in the defaults and will hopefully resolve itself in Mavericks.
: Mostly from https://community.qualys.com/blogs/securitylabs/2013/03/19/rc4-in-tls-is-br… and I’m not aware of any changes.