Mailman 3 April 2013 - Mailman-Developers

Re: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion]
by Stephen J. Turnbull 15 Apr '13

15 Apr '13

Sreyanth writes: > Just hide the boilerplate in the email, giving a link to > click. When clicked, use js to unhide the boilerplate. This would > not anyhow require separate storage. Suggest me something if this > is bad! The main point of sharing links is not storage compression; it's that the link identifies boilerplate you've seen before by changing appearance. You won't be able to tell if JS is used.

2 1

Re: [Mailman-Developers] anti-spam filter
by Pratik Sarkar 15 Apr '13

15 Apr '13

On Mon, Apr 15, 2013 at 2:19 AM, Pratik Sarkar <iampratiksarkar(a)gmail.com>wrote: > Yes,thats why the language model concept will come in handy. > The classifier will be trained on the existing archives. > For eg,in debian list,the classifier will be trained on a training set > containing copyright law,source code,etc.Whereas in an economists list,the > classifier will be trained on the training set containing economics-related > emails and conversations. > Whenever an economics-related message will try to enter the debian > list,the classifier would check the language model of the > (economics)message with the classifier and would classify it correctly as > the sentence construction/language model of a technical message(debian list > message) and an economics message varies a lot. > Please feel free to as me,if I have not been able to explain my views. > Regards > Pratik > > > On Mon, Apr 15, 2013 at 2:07 AM, Paul Wise <pabs3(a)bonedaddy.net> wrote: > >> On Mon, Apr 15, 2013 at 4:55 PM, Stephen J. Turnbull wrote: >> >> > That's a great idea (and a great example of why I don't think >> > spam-filtering technology should be embedded in a Mailman handler! :-) >> >> I think you might not be thinking expansively enough, it allows more >> flexibility to do spam filtering in multiple places. Different lists >> will have different definitions of what is an unwanted message. On the >> Debian lists we don't want anything about shares, liquidity and >> financial instruments, but some economists will find messages about >> source code and copyright law unwanted on their lists. A list for >> pharmacists will probably want to accept messages containing >> usually-spammy keywords. A generic mailing-list-hoster hosting all 3 >> of those will want to allow most mail and add per-list filters. >> >> -- >> bye, >> pabs >> >> http://bonedaddy.net/pabs3/ >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers(a)python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/iampratiksarkar%4… >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > >

1 0

Re: [Mailman-Developers] anti-spam filter
by Stephen J. Turnbull 15 Apr '13

15 Apr '13

Paul Wise writes: > One really useful thing to have would be removal of spam from > archives. Here are some examples of how it has been done before: That's a great idea (and a great example of why I don't think spam-filtering technology should be embedded in a Mailman handler! :-)

2 1

Re: [Mailman-Developers] Boilerplate and content filtering [was: Introduction and Project Discussion]
by Stephen J. Turnbull 15 Apr '13

15 Apr '13

Sreyanth writes: > I have implemented a binary Bayesian classifier which classifies an email > either spam or not spam. Is it better than (or at least different from) SpamBayes (available on PyPI), which some third parties already use in Mailman installations? Let's not reinvent wheels, here!

1 0

Re: [Mailman-Developers] anti-spam filter
by Paul Wise 15 Apr '13

15 Apr '13

On Mon, Apr 15, 2013 at 1:39 PM, Pratik Sarkar wrote: > Is the anti-spam/abuse filter still being seriously considered > as a gsoc project this year? One really useful thing to have would be removal of spam from archives. Here are some examples of how it has been done before: The Debian lists server isn't based on mailman but has a collaborative method of removing spam from the archive: https://wiki.debian.org/Teams/ListMaster/ListArchiveSpam Indymedia implemented a mailman 2 patch for this a number of years ago: https://lists.indymedia.org/patches/imc-10-mmid_hide_posts.patch -- bye, pabs http://bonedaddy.net/pabs3/

1 0

Boilerplate and content filtering [was: Introduction and Project Discussion]
by Stephen J. Turnbull 15 Apr '13

15 Apr '13

Sreyanth writes: > Also, I would like to hear more about : Boilerplate stripper AND Better > content-filtering / handling error messages. > Boilerplate stripping is trivial to understand. But, can anyone elaborate > on Better content-filtering / handling error messages? But boilerplate stripping is not necessarily trivial to implement, because it's not always clear what boilerplate is. I think it might be a good idea to save it off and provide a link rather than discard it, which leads to interesting questions of storage, shared links for true boilerplate (storage compression of repeatedly encountered text, yes, but more important the link will turn purple so you don't need to click on it in the next message from that user!), and user interface in general. Content filtering is mostly going to be about MIME handling: choice of the appropriate text/* part and things like that, removing images/video/etc where the list prohibits them, converting HTML/ wordprocessor attachments to plain text, removing MIME parts whose Content-Type doesn't match filename or perhaps file(1) magic in the content, etc. I can also imagine content filtering (or scoring!) based on word choice ("WTF" OK, spelling it out not :-). Also content filtering based on stripping out the quoted from top-posts and replacing them with links (after checking that the quoted material is indeed available in the archive!) All coming with on/off options, at least for those who remember the IBM 360saurus and other dinosaurs and still prefer mail to web. :-) Error messages (I think this means delivery status notifications (DSN) from mail servers) are a similar kind of problem to text-based filtering, though somewhat more stylized. Steve

3 3

anti-spam filter
by Pratik Sarkar 15 Apr '13

15 Apr '13

Hi guys, Is the anti-spam/abuse filter still being seriously considered as a gsoc project this year?

1 0

Wiki Migration Update
by Paul Boddie 14 Apr '13

14 Apr '13

Hello, It's been about a month or so since my last update, so here's a quick summary of the work done since that time on the content migration from Confluence to MoinMoin. First of all, I've uploaded a newer version of the content here: http://mmwiki.boddie.org.uk/ Not only has the conversion itself been improved, but the underlying Confluence content has also been updated, so it should reflect the existing Wiki (at http://wiki.list.org/) a bit more accurately. I also fought with mod_rewrite to at least make most pages accessible, particularly the FAQ-related pages which contain question marks in their titles (and with which mod_rewrite has an as-yet-unfixed problem). A lot of the recent work has gone into fixing the XHTML-based markup used by Confluence since version 4. In particular, links, lists, tables and preformatted regions have seen quite a bit of attention, and various pages should now look better due to this. For example: http://mmwiki.boddie.org.uk/DOC/Mailman_2.1_Members_Manual (Note that the above page has some fairly strange nested lists at the end where the bulletpoints have been added manually as "o" characters after newlines, and this is not an artifact of my conversion, but the markup is handled sufficiently well to retain the desired visual effect.) Some things that were mentioned as problem areas have been improved: http://mmwiki.boddie.org.uk/DEV/A_5_Minute_Guide_to_Get_the_Mailman_Web_UI_… (This now has properly preformatted regions, although I aim to improve the handling of single-line regions still further.) http://mmwiki.boddie.org.uk/COM/donate_to_the_GNU_Mailman_project (This now has a fixed link, although the Confluence markup version still needs improving to work correctly.) http://mmwiki.boddie.org.uk/DOC/Frequently_Asked_Questions (The links to questions now have tidier labels, and my mod_rewrite hacks mean that you can follow them on the hosted site.) A few things haven't yet been done since the last update that were worth mentioning before: http://mmwiki.boddie.org.uk/COM/Organizations_that_use_Mailman (I still need to make sure that the anchors are generated for the headings.) Author information is still not preserved in the page import process, but this will be tested in future. I want to start implementing macro support and to tidy up the way comments are presented on pages. It might also be nice to have a list of attachments on pages that have them, and I will take a look to see how Confluence tends to present such things. As I have mentioned previously, the source code for the converter can be found here: http://hgweb.boddie.org.uk/ConfluenceConverter/ Please take a look at your favourite pages and let me know where improvements can be made to the conversion process. Paul

1 0

Re: [Mailman-Developers] error installing the mailman web UI
by Avik Pal 14 Apr '13

14 Apr '13

at last resolved the issue, had to install *lazr.smtptest 2.0* separately, now as the the web UI is up and running its time to try out few things :) Avik Pal Bengal Engineering & Scieence University,Shibpur github:https://github.com/avikpal IRC:- irc://freenode/avikp,isnick twitter:-https://twitter.com/avikpalme On 14 April 2013 09:34, Avik Pal <avikpal.me(a)gmail.com> wrote: > actually I think the following is behind all these... > > *Download error on http://ish.io/projects/show/restish: [Errno -2] Name > or service not known -- Some packages may not be found!* > > Avik Pal > Bengal Engineering & Scieence University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 14 April 2013 01:52, Mark Sapiro <mark(a)msapiro.net> wrote: > >> Avik Pal wrote: >> >> >ya thanx sneha and sreyanth, hopfully other community members will be >> able >> >to help me out of this situation. >> >> >> The actual exception in your original post was a timeout trying to open >> an internet connection to retrieve some package. Are you behind some >> firewall or is there some other problem preventing you from connecting >> to PyPI or wherever it's going? >> >> -- >> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> >> >

1 0

GSOC Introduction and Bugfix Question
by Patrick Collins 14 Apr '13

14 Apr '13

Hello, My name is Patrick Collins and I'm a second year computer science major, interested in applying to GSOC via Mailman. I was thinking of taking on the following projects: 1) Boilerplate stripper 2) Log monitor I imagine that (1) would be pretty easy to implement and take no more than a week. (2) is more complex, but offers extensibility: I could aim to implement a log monitor that, at the bare minimum, warns the admins in case of trouble, and then go on to extend it with some visualizations like graphs that track subscribers/country, posts/subscriber, and so on. This would give admins the added benefit of being able to clearly see trends in the logs that indicate issues. Would this be be complex enough to merit a GSOC project? Additionally, I wanted to implement a simple bug fix in order to get familiar with the Launchpad system. https://bugs.launchpad.net/mailman/+bug/558274 seems to be a pretty quick one. Should I write tests to accompany my changes to the code, or just perform them on my own machine? Should there be some documentation of testing submitted? -- Patrick Collins University of Chicago '15

1 0