Mailman 3 April 2013 - Mailman-Developers

Architecture for extra profile info
by Terri Oda April 18, 2013

April 18, 2013

Background for folk new to this discussion: Currently, all user information is stored in Mailman core, but it's minimal: a real name, a set of email addresses, subscription info, and preferences. Barry suggests that it should stay minimal: only the things Mailman needs to know to correctly deliver mail (which actually doesn't include "real name" but let's leave that as a legacy item for the moment) It's pretty likely that future features of Mailman will want to attach extra information to users. Some of it will be social-y stuff like user icons for HyperKitty to display in the archives. Other things include metrics like "when did this person last post to the list?" or "how many posts have they made over the lifetime of this list?" One thing I know of is that Systers requires a short essay for all new subscribers, explaining why they want to join the list. (And they're considering porting this feature to Postorius, which means we potentially want an answer to "where will the extra profile data get stored?" before their students start coding.) So... I think we've sort of agreed that it would be best if whatever we built just had a rest interface and hyperkitty/postorius/whatever would talk to it through there, and could share data that way, but we need a simple prototype that folk (particularly students) will be able to start using, and there's still some internal architecture decisions that need to be made. Does anyone have time to build such a thing or write up some short architectural documents so a student could build such a thing in relatively short order? It doesn't have to be the perfect final design, but we probably need a basic starter api for adding, accessing, editing and possibly even removing profile data. Thoughts? Terri

3 2

Re: [Mailman-Developers] GSOC 2013 project discussion
by Stephen J. Turnbull April 18, 2013

April 18, 2013

Avik Pal writes: > Meanwhile It would be much appreciated if someone can direct me to > an labeled dataset available on line. By "labelled" you mean pre-classified into spam vs ham? I see you already found one, but you could also check the SpamBayes and SpamAssassin distributions. > Here I have a suggestion, after submitting, whenever an email is > classified as Spam, we store it in a separate archive and after the > end of the day send them a mail telling "this is the digest for all > the mails that Mailman thinks to be Spam" the subscriber may go > there and can view them and also can mark them as not Spam, I suggest that you present this as an option for users who want to tune the filters, and as something that can be used pre-release to develop the initial parameters for the distributed classifier. Although Bayesian classifiers do offer the option to train or tune your personal classifier on a local corpus, most users just stick with the distribution parameters plus self-training. It's pretty effective (surprisingly so to me). I guess the logic is that spammers aren't terribly creative. > Emails which stays as Spam will be dropped after a month Let's think carefully about that. Everybody deletes the spam; that's why you started by asking for a labelled dataset, because nobody keeps one around. Somebody really ought to do the public service of collecting a corpus. Of course, if you do arrange to keep it around, it's going to need to be an option that sites and list owners can disable.

2 1

Query regarding GSoC project
by Adwait Sharma April 17, 2013

April 17, 2013

Hello ! By way of introduction I am *Adwait Sharma,* a final year computer science undergraduate from Bangalore, India who code a lot specially in Python. The idea I found most interesting in GSoC 2013 is Boilerplate stripper - http://wiki.list.org/display/DEV/Google+Summer+of+Code+2013#GoogleSummerofC… I am planning to use regular expressions for this, obviously script will be in python :) Even though the project can be finished before the end of GSoC period. I would like to add *some more features* to it. 1. I am interested in adding Natural Language Processing techniques which will check for the word (attach, attached, attachment(s)) in the mail and will cross check if there is any file attached to it. If not, it will come up with a dialogue box saying, "*you forgot to attach a file*" 2. Apart from this, I would like to work on censorship as well which will work as a *dirty word filter*. and many more *Artificial Intelligence tools* in mailman. About me : I have *4 years* of programming experience in *Python*, using mailman extensively. I was a speaker at *PyCon India 2012 * http://in.pycon.org/2012/funnel/pyconindia2012/36-artificial-intelligence-u…. I would like to thank PSF for sponsoring me to *PyCon US 2013* where I took a lightning talk. I'm hoping to work with Mailman in this year's GSoC as I already have a experience of AI & NLP and this would serve as an great opportunity and chance to work with Mailman formally and prove my skills. I can be reached via email : *sharma(dot)adwait(at)gmail(dot)com* Looking forward to your reply. Kind regards, Adwait -- *Adwait Sharma ~NeO~* Learner | Geek | Hacker | Blogger | Open Source Software Developer Email: sharma.adwait(a)gmail.com Site: http://www.ThePirado.com <http://www.thepirado.com> [image: Blog RSS]<http://feeds.feedburner.com/TechLounge4u> Contact <http://www.facebook.com/sharma.adwait> <http://www.orkut.co.in/Main#Profile.aspx?uid=4511762959040111702> <http://twitter.com/adwait_sharma> <http://www.linkedin.com/pub/adwait-sharma/13/b46/849> <http://www.google.com/profiles/sharma.adwait> <http://www.blogger.com/profile/02982415035010183133> Phone: +91-9164665550

1 0

Re: [Mailman-Developers] GSOC 2013 project discussion
by Barry Warsaw April 17, 2013

April 17, 2013

On Apr 17, 2013, at 12:43 AM, Avik Pal wrote: >also I would like to propose an idea of my own. Many of us set the preference >in mailman to get all the emails of a day batched together, but sometimes >this means we miss important mails(though we get it at the end of the day but >we miss the moment)----important to the community, or my own interest, >discussion on something I also have discussed upon in my previous mails, >delivery of these mails instantly to the subscriber so that he can also join >at that very moment may come out to be a very useful feature. I'm skeptical that you'll be able to do automated classification of important messages for individual users. I'm not even sure I can do that as a human reading a particular mailing list. Also, what's important to me today may or may not be important to me tomorrow, and vice versa. Mailman has a (probably little known) feature where a list owner can send an urgent message that bypasses digests. If it has an Urgent: header with a matching password, it goes straight through. -Barry

1 0

Re: [Mailman-Developers] GSOC 2013 project discussion
by Terri Oda April 17, 2013

April 17, 2013

I'm glad you're somewhat aware of the issues. I frequently encounter folk who aren't aware of the issues in machine learning, so your "don't lose hope" email set off all kinds of warning bells in my head. Going back to GSoC-specific stuff: - Enron is a very old data set - If you're going to use it, you need to be prepared to defend that choice. I'm not sure it's a choice that can be defended at all, knowing the field. It's probably not only an old data set, but a completely counter-productive one given the space in which Mailman operates. So here's some things to think about: (1) I want some justification of how this is going to be relevant to the problem you're trying to solve, which is "helping classify spam emails sent to a mailing list that the MTA was unable to classify" (2) Many existing classifiers that run at the MTA level have already used the enron data set, so chances are any features you learn will either already have been incorporated. I have severe concerns that any new features you learn will result in over-fitting. How can you believe that yet another classifier trained on the same data will be worth the processing overhead and resulting delays in mail delivery when it seems likely that any improvement will be incremental at best? (3) Enron is not going to help you make use of any list-specific features. How can you use this data set to produce something that is useful to Mailman, going beyond what any MTA-level spam filter can do? (Note that we've been telling people to do spam filtering at the MTA level for years and years and years; justifying this is not going to be an easy task) (4) If you're going to do cross-validation with other data to make claims that the final classifier will be relevant to list data, how is that data going to be obtained, processed, and used? (5) Unless you've got a plan for making extensive use of the fact that you're classifying mailing list data and not general email, you're pretty much wasting our time since we are only interested in projects relevant to Mailman. To be completely honest, I'm still seeing "student project for data mining class" level thinking here, and that's not going to be good enough for us. Especially considering that you didn't even know about the most common data sets for this problem, I'm concerned that you haven't yet reached the skill and experience necessary for us to seriously consider a classifier as even a small part of a GSoC project. We have to give priority to students who we are convinced can finish their projects, and it seems like there's too many chances of you getting stuck on finding data and using it correctly on a problem that is actually meaningful to Mailman and not just a general classification task. Terri On 13-04-17 10:51 AM, Avik Pal wrote: > ya I get your point, but see these are part of any machine learning > project, and feature extraction has to be done considering the > synthetic data set. > > > On 17 April 2013 22:05, Terri Oda <terri(a)zone12.com > <mailto:terri@zone12.com>> wrote: > > > > Finding sources of spam (like that one) isn't that hard; it's > finding sources of legit email combined with spam and classified > and processed in the same way that's challenging. As I said, you > can combine a spam source like this with a publicly available > mailing list to make a synthetic set, but scientifically speaking, > those aren't really preferred ways to handle data because they > come from multiple sources. > > > well in this regard the only thing I can do is keep looking, I am > also aware that coming from different sources can make them skewed but > again these things are never perfect and there are always scope for > betterment, I think that our aim should be to implement a rudimentary > classifier with fairly good performance to start with.

1 0

Re: [Mailman-Developers] GSOC 2013 project discussion
by Terri Oda April 17, 2013

April 17, 2013

On 13-04-17 10:10 AM, Avik Pal wrote: > Don't lose hope Terri, after digging for a couple of hours came across > this and its pretty much updated. http://untroubled.org/spam/ Finding sources of spam (like that one) isn't that hard; it's finding sources of legit email combined with spam and classified and processed in the same way that's challenging. As I said, you can combine a spam source like this with a publicly available mailing list to make a synthetic set, but scientifically speaking, those aren't really preferred ways to handle data because they come from multiple sources. The problem is that when you have multiple sources it sometimes becomes too easy for a classifier to classify on less-than-useful features for future use. For example, one might classify on the fact that the list address won't appear in any of the To: or Cc: lines in the spam data because it comes from a different source, the fact that many of the spams will be from different time periods, the fact that the spam data is anonymized differently from any list data you might have, etc. You will wind up doing a lot of work to normalize the data sets to avoid these classifiers (and we're talking weeks of really boring work here, potentially, that you need to start Right Now if you're going to be using such a set), and you run the risk of missing out on features that would have been useful in a single-source set that have been completely obliterated by the synthetic data set. Terri

2 1

Re: [Mailman-Developers] GSOC 2013 project discussion
by Avik Pal April 17, 2013

April 17, 2013

> On 17 April 2013 21:02, Terri Oda <terri(a)zone12.com> wrote: > >> >> On 13-04-17 6:56 AM, Avik Pal wrote: >> >>> Meanwhile It would be much appreciated if someone can direct >>> me to >>> an labeled dataset available on line. >>> >>> Leaving aside entirely the question of whether we should (or will) >> support any project that requires learning on this scale, as a former >> anti-spam researcher, I can at least answer this question. >> >> Unfortunately, the answer is largely "good luck with that" -- good >> labelled email data is surprisingly hard to come by, and that challenge is >> one of the reasons I stopped doing research in that area. >> >> >> > > Don't lose hope Terri, after digging for a couple of hours came across this and its pretty much updated. http://untroubled.org/spam/

1 0

Re: [Mailman-Developers] GSOC 2013 project discussion
by Terri Oda April 17, 2013

April 17, 2013

On 13-04-17 6:56 AM, Avik Pal wrote: > Meanwhile It would be much appreciated if someone can direct me to > an labeled dataset available on line. > Leaving aside entirely the question of whether we should (or will) support any project that requires learning on this scale, as a former anti-spam researcher, I can at least answer this question. Unfortunately, the answer is largely "good luck with that" -- good labelled email data is surprisingly hard to come by, and that challenge is one of the reasons I stopped doing research in that area. When I was doing anti-spam research, the only viable public classified ham/spam set was the SpamAssassin one. I don't believe it's been maintained with modern messages and at this point it may be useless. Shortly after I left the field, people started using the Enron data set, which is pretty well classified by now, but again, is pretty long in the tooth. Given that you're going to want to be classifying mailing list data, you may have to produce some synthetic data sets using information from publicly available mailing lists (e.g. the public archives of mailman-developers are available) and combining them with other data sources (e.g. publicly available collections of spam). This won't have a whole lot of interesting sub-labels (some lists will have more than others, depending on their use of dlists/topics/pre-classification by the sender) and a synthetic set is generally regarded as a poor information source for reproducible results, but it could be enough in a pinch given that you're adding a feature rather than publishing scientific work. Note that the GSoC timeline doesn't allow time for finding and creating such a set, so if you're going to use one, you should determine in advance what you'll be using and and be able to provide a link to the completely-ready-for-gsoc set in your proposal. Terri

2 1

Re: [Mailman-Developers] GSOC 2013 project discussion
by Richard Wackerbarth April 17, 2013

April 17, 2013

In evaluating a proposal, we need to look at a number of factors: First, will it work? -- Does the proposed design accomplish the stated objective? Next: Is it useful? And: Can the candidate be expected to accomplish the task within the allotted time frame? Finally: Is it the best use for our limited resources (funding, mentor time, etc.)? If your presentation makes it easier to answer each of those questions in a positive manner, it will increase the likelihood that it will get funded. On Apr 17, 2013, at 6:16 AM, Avik Pal <avikpal.me(a)gmail.com> wrote: > for identifying an important message a classifier will be implemented. and thanks for pointing out the issue regarding the delivery of the message, if it is delivered twice then the existing implementation of delivery is sufficient, but if we want to deliver it only once then for each person we need to maintain a database of important mails/threads to him(or vice-versa) and while sending check against that database. but this is going to raise some normalization issues which are to be taken care of by careful designing. > > Avik Pal > Bengal Engineering & Science University,Shibpur > github:https://github.com/avikpal > IRC:- irc://freenode/avikp,isnick > twitter:-https://twitter.com/avikpalme > > > > > > On 17 April 2013 01:02, Richard Wackerbarth <rkw(a)dataplex.net> wrote: > An interesting suggestion -- A couple of things to consider: > > How do you identify "important" messages? > > Will you deliver these messages twice -- first as important and then, later, as a part of the digest ? > > > On Apr 16, 2013, at 2:13 PM, Avik Pal <avikpal.me(a)gmail.com> wrote: > > also I would like to propose an idea of my own. Many of us set the > > preference in mailman to get all the emails of a day batched together, but > > sometimes this means we miss important mails(though we get it at the end of > > the day but we miss the moment)----important to the community, or my own > > interest, discussion on something I also have discussed upon in my previous > > mails, delivery of these mails instantly to the subscriber so that he can > > also join at that very moment may come out to be a very useful feature. > > Thus person gets to set two options > > 1.receive batched mails only. > > 2.receive batched mails with important mails delivered instantly. > >

2 1

Re: [Mailman-Developers] GSOC 2013 project discussion
by Richard Wackerbarth April 17, 2013

April 17, 2013

An interesting suggestion -- A couple of things to consider: How do you identify "important" messages? Will you deliver these messages twice -- first as important and then, later, as a part of the digest ? On Apr 16, 2013, at 2:13 PM, Avik Pal <avikpal.me(a)gmail.com> wrote: > also I would like to propose an idea of my own. Many of us set the > preference in mailman to get all the emails of a day batched together, but > sometimes this means we miss important mails(though we get it at the end of > the day but we miss the moment)----important to the community, or my own > interest, discussion on something I also have discussed upon in my previous > mails, delivery of these mails instantly to the subscriber so that he can > also join at that very moment may come out to be a very useful feature. > Thus person gets to set two options > 1.receive batched mails only. > 2.receive batched mails with important mails delivered instantly.

2 1