[Twisted-Python] Status of trac upgrade
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
Hello, I was just wondering what the current status of the effort to upgrade trac on twistedmatrix.com is. Thanks, Jean-Paul
![](https://secure.gravatar.com/avatar/174e7b0ff60963f821d0b9a4f1a3ef52.jpg?s=120&d=mm&r=g)
On 23 May 2014, at 15:11, exarkun@twistedmatrix.com wrote:
I was just wondering what the current status of the effort to upgrade trac on twistedmatrix.com is.
I could hear the crickets all the way down to Madagascar. So what *is* the status? The current state is really hardly bearable; the spam is taking completely over. :( Wasn’t there a successful dry run at the PyCon sprints? Cheers Hynek *** Also JFTR and related to our old plans to utilize GitHub somehow: it seems like Phabricator would be much rather worth our time as it allows for a complete review workflow: http://cramer.io/2014/05/03/on-pull-requests/
![](https://secure.gravatar.com/avatar/13789173efc8ea92ddd65b23535937d4.jpg?s=120&d=mm&r=g)
On Thu, May 29, 2014 at 03:13:01PM +0200, Hynek Schlawack wrote:
For this matter: I expressed a few times my interest here to help Twisted move to Git, and upgrading Trac was a requirement. Although I basically don't have any special rights in the Twisted project (oh, I can edit the wiki now!), I would be happy to help in any ways to complete this migration, I'm just not sure what I can do, and how it can be done. However, I really won't be able to do anything before end of July, but after that, I will be able to dedicate time to help on this, and hopefully the Git migration (or things connected to) as well... Jonathan
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 29, 2014, at 8:12 AM, Dustin J. Mitchell <dustin@v.igoro.us> wrote:
It took a long time, but it was definitely worth it!
Thanks for this input, Dustin. It is actually _super_ useful, for me at least, to learn that there is in fact a light at the end of this tunnel :). -glyph
![](https://secure.gravatar.com/avatar/d37c7104c024b78dc451e3a6b733df9d.jpg?s=120&d=mm&r=g)
On May 29, 2014, at 9:13 AM, Hynek Schlawack wrote:
So what *is* the status? The current state is really hardly bearable; the spam is taking completely over. :( Wasn’t there a successful dry run at the PyCon sprints?
I recently had a similar problem. I didn't realize a "one click install" on my shared provider for a private SVN repo created a public trac instance. there were nearly 1MM spam tickets in a 700MB sqlite database I ended up killing all tickets; but was able to use a raw sqlite3 connection on the db file to get in there and analyze the tickets ( and delete them ) Trac 1.0 has a spam filter -- http://trac.edgewall.org/wiki/SpamFilter Once upon a time, there was a mod_security plugin called ScallyWhack that was dedicated to Trac spam. It was officially supported by mod_security and still has a reserved rules range. unfortunately, it's disappeared off the net. I had to take my trac instance offline while working. my install was "known" to a few dozen botnets, and they kept hitting it. everything would lock up. if you can find any mod_security integration, I would strongly suggest using it -- because you can have the rules trigger an integration with fail_2_ban and just keep ip addresses/ranges from ever touching trac.
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
On 18 Jun, 10:59 pm, twisted-python@2xlp.com wrote:
This is a nice thought but I think it's entirely misguided. Overcoming simplistic, automated obstacles is what spammers have been learning how to do extremely well for several decades now. If you choose to participate in this arms race with them, you can win by put in slightly more effort than them - from now until forever. Considering the Twisted project apparently lacks even the ability to put a slight bit of effort even once (at least, not without gathering its strength to do so for two or three months first), this doesn't strike me as likely to happen. Also, Apache isn't used anywhere on twistedmatrix.com so it would be rather difficult to deploy anything based on mod_security anyway. Jean-Paul
![](https://secure.gravatar.com/avatar/d37c7104c024b78dc451e3a6b733df9d.jpg?s=120&d=mm&r=g)
On Jun 18, 2014, at 8:43 PM, exarkun@twistedmatrix.com wrote:
This is a nice thought but I think it's entirely misguided.
Overcoming simplistic, automated obstacles is what spammers have been learning how to do extremely well for several decades now. If you choose to participate in this arms race with them, you can win by put in slightly more effort than them - from now until forever.
I generally agree. Spam will always get through. But if you put a tiny amount of effort in, you can effectively block 99% of spam; and make working with that 1% much easier. There are a lot of smart spammers; there are more stupid and lazy ones. Spammers are generally smart at innovating over long periods of time, but their short-term attacks are pretty bad. If something gets noticed as a possible spam target by a network of compromised machines, the attacks are relentless. One particular IP block hits a particular trac install of mine every 5 seconds to make new tickets and check old ones -- even weeks after I configured their entire network to 403. If you keep some rules updated, and can integrate fail_2_ban, that entire process is automated.
Also, Apache isn't used anywhere on twistedmatrix.com so it would be rather difficult to deploy anything based on mod_security anyway.
I don't use Apache either. I use nginx as a frontend gateway to pass back to Twisted / Pyramid / etc. There's been mod_security support for nginx ( java and iis too ) for a few years. But trac is going through TwistedWeb/11.1.0, so that point is moot. But if you ever run a project that uses nginx on the front, you can use mod_security on it. The install is a bit weird, but it works. ... In any event, turning off trac and using raw sql queries was the best route to managing the trac database. If you pay attention to your server logs to see which ip addresses hit the "create" and "view" tickets a lot, you'll probably note a few IP blocks that have seem to have a "pair" of spiders working together on different machines. one creates spam, the other harvests tickets for email addresses. there were a handful of servers in the 96.47.2xx.x space responsible for most of my spam. i'd say probably 80%. The specific ips all ranked high on the Honeypot blacklist with hundreds of thousands of reports -- http://www.projecthoneypot.org/list_of_ips.php
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
On 23 May, 01:11 pm, exarkun@twistedmatrix.com wrote:
Hello again, Lacking any updates on this, I make some adjustments today. Authenticated users no longer automatically have permission to create or modify tickets. I imagine the next step in this direction would be to communicate this information to users automatically and give them instructions about how to be granted these permissions. If anyone has any suggestions about how this could be done, they would be quite welcome. And if anyone wants to continue the upgrade effort then of course that would be appreciated as well. Thanks, Jean-Paul
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jun 16, 2014, at 1:34 PM, Adi Roiban <adi@roiban.ro> wrote:
Hi Adi, Mișu, Nobody's really in charge of this per se, but David Reid did some work on it earlier. He and I will be trying to do some work on it this Thursday to move it forward and particularly to put more information on the relevant PR <https://github.com/twisted-infra/trac-config/pull/10> to try to make it possible for folks like yourself to contribute. If you'd like to be around in #twisted-admin on Freenode when we're doing that (starting approximately 10AM PST) we'll try to be there. Thanks for your interest, -glyph
![](https://secure.gravatar.com/avatar/174e7b0ff60963f821d0b9a4f1a3ef52.jpg?s=120&d=mm&r=g)
On 23 May 2014, at 15:11, exarkun@twistedmatrix.com wrote:
I was just wondering what the current status of the effort to upgrade trac on twistedmatrix.com is.
I could hear the crickets all the way down to Madagascar. So what *is* the status? The current state is really hardly bearable; the spam is taking completely over. :( Wasn’t there a successful dry run at the PyCon sprints? Cheers Hynek *** Also JFTR and related to our old plans to utilize GitHub somehow: it seems like Phabricator would be much rather worth our time as it allows for a complete review workflow: http://cramer.io/2014/05/03/on-pull-requests/
![](https://secure.gravatar.com/avatar/13789173efc8ea92ddd65b23535937d4.jpg?s=120&d=mm&r=g)
On Thu, May 29, 2014 at 03:13:01PM +0200, Hynek Schlawack wrote:
For this matter: I expressed a few times my interest here to help Twisted move to Git, and upgrading Trac was a requirement. Although I basically don't have any special rights in the Twisted project (oh, I can edit the wiki now!), I would be happy to help in any ways to complete this migration, I'm just not sure what I can do, and how it can be done. However, I really won't be able to do anything before end of July, but after that, I will be able to dedicate time to help on this, and hopefully the Git migration (or things connected to) as well... Jonathan
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On May 29, 2014, at 8:12 AM, Dustin J. Mitchell <dustin@v.igoro.us> wrote:
It took a long time, but it was definitely worth it!
Thanks for this input, Dustin. It is actually _super_ useful, for me at least, to learn that there is in fact a light at the end of this tunnel :). -glyph
![](https://secure.gravatar.com/avatar/d37c7104c024b78dc451e3a6b733df9d.jpg?s=120&d=mm&r=g)
On May 29, 2014, at 9:13 AM, Hynek Schlawack wrote:
So what *is* the status? The current state is really hardly bearable; the spam is taking completely over. :( Wasn’t there a successful dry run at the PyCon sprints?
I recently had a similar problem. I didn't realize a "one click install" on my shared provider for a private SVN repo created a public trac instance. there were nearly 1MM spam tickets in a 700MB sqlite database I ended up killing all tickets; but was able to use a raw sqlite3 connection on the db file to get in there and analyze the tickets ( and delete them ) Trac 1.0 has a spam filter -- http://trac.edgewall.org/wiki/SpamFilter Once upon a time, there was a mod_security plugin called ScallyWhack that was dedicated to Trac spam. It was officially supported by mod_security and still has a reserved rules range. unfortunately, it's disappeared off the net. I had to take my trac instance offline while working. my install was "known" to a few dozen botnets, and they kept hitting it. everything would lock up. if you can find any mod_security integration, I would strongly suggest using it -- because you can have the rules trigger an integration with fail_2_ban and just keep ip addresses/ranges from ever touching trac.
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
On 18 Jun, 10:59 pm, twisted-python@2xlp.com wrote:
This is a nice thought but I think it's entirely misguided. Overcoming simplistic, automated obstacles is what spammers have been learning how to do extremely well for several decades now. If you choose to participate in this arms race with them, you can win by put in slightly more effort than them - from now until forever. Considering the Twisted project apparently lacks even the ability to put a slight bit of effort even once (at least, not without gathering its strength to do so for two or three months first), this doesn't strike me as likely to happen. Also, Apache isn't used anywhere on twistedmatrix.com so it would be rather difficult to deploy anything based on mod_security anyway. Jean-Paul
![](https://secure.gravatar.com/avatar/d37c7104c024b78dc451e3a6b733df9d.jpg?s=120&d=mm&r=g)
On Jun 18, 2014, at 8:43 PM, exarkun@twistedmatrix.com wrote:
This is a nice thought but I think it's entirely misguided.
Overcoming simplistic, automated obstacles is what spammers have been learning how to do extremely well for several decades now. If you choose to participate in this arms race with them, you can win by put in slightly more effort than them - from now until forever.
I generally agree. Spam will always get through. But if you put a tiny amount of effort in, you can effectively block 99% of spam; and make working with that 1% much easier. There are a lot of smart spammers; there are more stupid and lazy ones. Spammers are generally smart at innovating over long periods of time, but their short-term attacks are pretty bad. If something gets noticed as a possible spam target by a network of compromised machines, the attacks are relentless. One particular IP block hits a particular trac install of mine every 5 seconds to make new tickets and check old ones -- even weeks after I configured their entire network to 403. If you keep some rules updated, and can integrate fail_2_ban, that entire process is automated.
Also, Apache isn't used anywhere on twistedmatrix.com so it would be rather difficult to deploy anything based on mod_security anyway.
I don't use Apache either. I use nginx as a frontend gateway to pass back to Twisted / Pyramid / etc. There's been mod_security support for nginx ( java and iis too ) for a few years. But trac is going through TwistedWeb/11.1.0, so that point is moot. But if you ever run a project that uses nginx on the front, you can use mod_security on it. The install is a bit weird, but it works. ... In any event, turning off trac and using raw sql queries was the best route to managing the trac database. If you pay attention to your server logs to see which ip addresses hit the "create" and "view" tickets a lot, you'll probably note a few IP blocks that have seem to have a "pair" of spiders working together on different machines. one creates spam, the other harvests tickets for email addresses. there were a handful of servers in the 96.47.2xx.x space responsible for most of my spam. i'd say probably 80%. The specific ips all ranked high on the Honeypot blacklist with hundreds of thousands of reports -- http://www.projecthoneypot.org/list_of_ips.php
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
On 23 May, 01:11 pm, exarkun@twistedmatrix.com wrote:
Hello again, Lacking any updates on this, I make some adjustments today. Authenticated users no longer automatically have permission to create or modify tickets. I imagine the next step in this direction would be to communicate this information to users automatically and give them instructions about how to be granted these permissions. If anyone has any suggestions about how this could be done, they would be quite welcome. And if anyone wants to continue the upgrade effort then of course that would be appreciated as well. Thanks, Jean-Paul
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jun 16, 2014, at 1:34 PM, Adi Roiban <adi@roiban.ro> wrote:
Hi Adi, Mișu, Nobody's really in charge of this per se, but David Reid did some work on it earlier. He and I will be trying to do some work on it this Thursday to move it forward and particularly to put more information on the relevant PR <https://github.com/twisted-infra/trac-config/pull/10> to try to make it possible for folks like yourself to contribute. If you'd like to be around in #twisted-admin on Freenode when we're doing that (starting approximately 10AM PST) we'll try to be there. Thanks for your interest, -glyph
participants (8)
-
Adi Roiban
-
Dustin J. Mitchell
-
exarkun@twistedmatrix.com
-
Glyph
-
Glyph Lefkowitz
-
Hynek Schlawack
-
Jonathan Ballet
-
Jonathan Vanasco