Mailman 3 March 2016 - Mailman-Developers

Re: [Mailman-Developers] [Mailman-Developters] GSoC support
by Abhilash Raj 06 Mar '16

06 Mar '16

Hi Amil, On 03/05/2016 03:22 AM, amil shanaka wrote: > Hi, > I'm a 3rd year under graduate in the department of Computer Science and > Engineering at University of Moratuwa, Sri Lanka > > I tried to connect with mailing list in Mainman but it doesn't work. > > I did my Internship In WSO2 Message Broker Team. so i learned about > MQTT(with netty) and AMQP protocols about how message broker work with > sub/pub with topic,durable topics and also i did a research about off-heap > implementation for mb. > > So I want to know more details regarding* Message queue based email > archiver *Project which i can proceed through. Right now, Mailman sends each email it receives to the (enabled) Archivers through a POST request. We want to have a pub/sub kind of model for the emails as well as some of the events that other archivers or apps built on top of mailman can consume. Events like ListCreated, ListRemoved or something like that. The above is just a very basic idea of what we want. You need to focus your application around how we can do this. Florian and Aurelien would be able to help you better if you come up with some ideas. You might want to go through the current codebase for Mailman, looking for the parts of code that are concerned with Archivers. A general idea about the complete architecture would also be a good idea. There are several design documentation about the architecture of Mailman 3 core. You can find article in the mailman3 docs, there is a chapter in Architecture of Open Source Applications book v2 and also there is a talk that Barry gave in PyCon 2012. Just as a reminder, you need to submit atleast one merge request fixing any of the issues mentioned in the bug tracker of any mailman projects. Let us know if you have more questions. > > Please Refer My LinkedIn: (amil shanaka > <https://lk.linkedin.com/in/amilshanaka>) for more detail about me and the > projects I've done so far. > > Hope you will help me with this. > > Thanks, > -- thanks, Abhilash Raj

2 1

Re: [Mailman-Developers] CAPTCHA support
by David Andrews 06 Mar '16

06 Mar '16

At 04:57 AM 3/5/2016, Aditya Divekar wrote: >Hi! >I was looking around the mailman code, and could not find the functionality >for captcha in the mailing lists subscription pages. >I think it could be a good feature to implement in the upcoming versions, >and would like to know if its a good idea? >I also came across the thread - >http://permalink.gmane.org/gmane.mail.mailman.user/74167 >and read about the previous discussion on it by some members. >Since captcha back then was easier to break, it might not have been a >profitable feature, according to the thread. >But with the new recaptcha, I would like to know if the stand is the same. There are a number of blind Mailman Admins out here, including myself, and many thousands of b lind or visually impaired users, for whom CAPTCHA can be a problem. We hate pictures of words and numbers, and are in favor of no CAPTCHA! Dave David Andrews and long white cane Harry. E-Mail: dandrews(a)visi.com or david.andrews(a)nfbnet.org

1 0

Re: [Mailman-Developers] CAPTCHA support
by Rich Kulawiec 06 Mar '16

06 Mar '16

On Sat, Mar 05, 2016 at 04:27:31PM +0530, Aditya Divekar wrote: > I was looking around the mailman code, and could not find the functionality > for captcha in the mailing lists subscription pages. As someone who has been studying email abuse for 30+ years, I strongly recommend against captchas for several reasons. First, as noted elsewhere in this thread, they're problematic for impaired or disabled users. Second, they've been quite thoroughly defeated by advances in image processing and character recognition. We have long since passed the point where the difficulty of captchas solvable by software has exceeded the difficulty of captchas solvable by humans. Third, as often noted elsewhere, it is relatively easy to conscript humans (knowingly or unknowingly) into the mass solving of captchas. Fourth, either a given instance is or is not a target of interest to adversaries. If it is not, the captchas are of course not needed. If it is, then they will not help: any modestly-clueful adversary will go through them like they're not even there. Bottom line: captchas are, at best, wishful thinking. There is zero operational reason to deploy them in 2016. ---rsk

1 0

CAPTCHA support
by Stephen J. Turnbull 06 Mar '16

06 Mar '16

Aditya Divekar writes: > But with the new recaptcha, I would like to know if the stand is > the same. ReCAPTCHA looks very good, though I haven't used it. However, the professional spammers are quite capable of hiring humans to do their work, and even conceal it as being charitable work (digitizing old books, for example). On the other hand, unsophisticated spammers are easily turned away with simple textchas (eg, the Python sites use questions like "What is van Rossum's first name?" -- no, I didn't try "Just" :-). I conclude that spam is a disease and evolves resistance to antispam technology at rates that resemble the ability of bacteria to evolve resistance to antibiotics, and we just have to keep a careful watch on every mail service we administer to see what the spammers are up to in our neighborhoods. Steve

1 0

Basic workflow of the ARC implementation
by Aditya Divekar 05 Mar '16

05 Mar '16

Hi Steve! Now going back to the implementation, the message needs to be signed after its headers have been cooked, since they are included in some of the signatures below. So after the message headers have been cooked, and before the message has been enqueued by the switchboards onto the various queues, we shall catch the email, and perform the ARC operations on it as given below. After this we can reinject the mail into the queues. This can be done by modifying the process that handles the queuing of the mails.(This can be done when the integration process into Mailman starts later on) For now, I will assume that only the message is available to us in its complete form after the list has cooked its headers. We need to generate a private and a public key for the signing purposes. For testing purposes, and while working on the code, I can probably generate the keys locally using the openssl tool. As a rough sketch of implementation, 1. ARC Seal The tags - NOTE - Now, here we needed to check the given message for any pre-existing signatures in most of the fields. For this I think a separate module can be created which can extract the previous ARC headers if they exist from the message. The code for this can be again used from the dkimpy package. i: The value for this tag can be determined by performing a check on the original signature and seeing if there were previous ARC headers. If yes, we increment the value of the previous "i" by 1, and if no, make it 1. a: The value for this is fixed. t,s,d: These can be obtained by using pre-existing dkimpy package. b: Now we can compute the header hash using the dkimpy package again by using the headers given here <https://tools.ietf.org/html/draft-andersen-arc-02#section-5.1.1.3>. Here, we call the dkimpy package and get the signature for the above headers and then affix it to the "b="header. cv: Use the same check as "i", if there is already an ARC i.e. i>1, then we make it as "V", else "N". The ARC Seal gets computed here. NOTE-For giving the "s" and "d" (selector and domain tag values), we will need to produce records for these where the key can be stored so that it is available for query by the verifiers (I still have to look up this mechanism). 2. ARC Message Signature The tags - i: The value for this tag is determined similar to the "i" tag for the ARC Seal. a,t,s,d: These can again be obtained from the dkimpy package. bh: The body hash. This can be obtained from the package. Here, we set the canonicalization to 'relaxed' and get the body hash. h: The "h=" header list is signed with the implicit list (as given in the draft) and any explicit list that we want in addition. Now, for mailing lists, the recommended headers are - List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:Reply-To: and any other fields added by the list like the Precedence, XTopics or else. We will also sign the DKIM signature of the previous mail here if available(suggested by the draft). For this signing, we can use a modified version where the FROZEN_HEADERS(headers that are signed by default) will specify the implicit headers according to ARC specs(Another option could be to get all the implicit+explicit headers signed by the package, extract the h header, and modify it to include only the explicit headers.) b: The signature, calculated from the package. The h tag is used as described above. 3. ARC Authentication Results. The "i" tag simply takes on the value same as the above "i" tags. Now from our previous conversation, as you suggested, the authenticity of the previous MTA who sent us the mail is not sure to be trusted. So in the case where we don't trust the previous MTA, we will have to perform our own DMARC, SPF, DKIM testing of the recieved mail. If previous ARC chain exists i.e. cv=V, then we perform the ARC test too. Now for performing the tests - In one of the earlier mails, we discussed the use of the "authres" package for generating the authentication results header. The package conforms to the RFC7001 format, and now the format used is RFC7601. But according to the changes that I verified, we can use the package without any changes. (The changes were mostly related to extra specifications that are optional. Can be skipped for our purposes) So the "authres" package can be called here for generating the AAR. If we need to perform the ARC test, then the module for that will have to be implemented manually. Though most of the code from the package for DKIM verification can be used. This is also the point where we detect if the mail is spam or not. If the arc test fails, then there is something fishy here. DKIM, DMARC, SPF may fail, but the failure of this test means the mail is not authentic. At this point the mail should probably be discarded (or any other measures that need to be taken). Now coming to the testing part. There can be a number of tests like verifying the generated ARC signature, changing the body of the message, failing when the implicitly signed AMS headers are changed and other such tests. Is this the workflow you were expecting to see, or should I write a more explanatory draft ? I would like to know your review of this, so that I can figure out if I am going wrong anywhere. :) (Also wherever I have mentioned the use of dkimpy, a lot of custom implementation is needed to suit our requirements. ) Thanks! Aditya

1 0

[Mailman-Developters] GSoC support
by amil shanaka 05 Mar '16

05 Mar '16

Hi, I'm a 3rd year under graduate in the department of Computer Science and Engineering at University of Moratuwa, Sri Lanka I tried to connect with mailing list in Mainman but it doesn't work. I did my Internship In WSO2 Message Broker Team. so i learned about MQTT(with netty) and AMQP protocols about how message broker work with sub/pub with topic,durable topics and also i did a research about off-heap implementation for mb. So I want to know more details regarding* Message queue based email archiver *Project which i can proceed through. Please Refer My LinkedIn: (amil shanaka <https://lk.linkedin.com/in/amilshanaka>) for more detail about me and the projects I've done so far. Hope you will help me with this. Thanks, -- *Amil shanaka* *Integrated Computer Engineering | University Of Moratuwa* mobile : (+94) *718548973 *| email : *catchamil(a)gmail.com* <catchamil(a)gmail.com> | *amil.12(a)cse.mrt.ac.lk <amil.12(a)cse.mrt.ac.lk>* skype : *amilshanaka*

1 0

CAPTCHA support
by Aditya Divekar 05 Mar '16

05 Mar '16

Hi! I was looking around the mailman code, and could not find the functionality for captcha in the mailing lists subscription pages. I think it could be a good feature to implement in the upcoming versions, and would like to know if its a good idea? I also came across the thread - http://permalink.gmane.org/gmane.mail.mailman.user/74167 and read about the previous discussion on it by some members. Since captcha back then was easier to break, it might not have been a profitable feature, according to the thread. But with the new recaptcha, I would like to know if the stand is the same. Thanks Aditya

1 0

Re: [Mailman-Developers] GSOC 2016 - Contribution in Mailman
by Abhilash Raj 04 Mar '16

04 Mar '16

Hi Sanjana, On 03/04/2016 12:07 AM, Sanjana Garg wrote: > Hello, > > I am Sanjana Garg, a third year undergraduate in the Department of Computer > Science and Engineering in IIT Kanpur. I am interesting in taking up the > project "Preset list setting templates". Welcome! As I have pointed students before to an older ongoing discussion on this particular topic, I am gonna do the same for you. Here[1] is the link for the thread that has been going on. You can go through the discussion and let me know if you have questions or new ideas regarding anything. > I am proficient in python. It would be great if someone could guide me for > the intial tasks required for the same. The GSoC wiki page talks extensively about how to get started with setting up your development environment and working on some (at least one for application) issues. Links to appropriate documentation and issue trackers are also provided. [1]: http://www.mail-archive.com/mailman-developers%40python.org/msg16023.html > Thanks > Sanjana Garg > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers(a)python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40… > > Security Policy: http://wiki.list.org/x/QIA9 > -- thanks, Abhilash Raj

1 0

GSOC 2016 - Contribution in Mailman
by Sanjana Garg 04 Mar '16

04 Mar '16

Hello, I am Sanjana Garg, a third year undergraduate in the Department of Computer Science and Engineering in IIT Kanpur. I am interesting in taking up the project "Preset list setting templates". I am proficient in python. It would be great if someone could guide me for the intial tasks required for the same. Thanks Sanjana Garg

1 0

Re: [Mailman-Developers] ARC module implementation [was: GSOC 2016]
by Stephen J. Turnbull 04 Mar '16

04 Mar '16

Aditya Divekar writes: > Since the receiver is going to get the mail through the mailing > list anyway, personally I think that the From munging strategy is > not a bad idea. Yeah, everybody who advocates it "personally thinks" it's a not bad strategy. The problem is that it out-and-out violates the RFCs, general ethics, and possibly copyright law, unless you think that forwarding something verbatim with possible addition of non-editorial content (eg, list serial numbers and subscription information in the footer) makes you an author. There's no other use case I can think of where people seriously maintain that editors are authors. One minor UI problem is that some MUAs can and are configured to display a personal icon rather than the display name -- choice of which is usually based on the email address. To my mind, though, the big problem is that from-munging does cause a number of technical problems, such as handling of reply-to. There simply is no satisfactory answer for all use cases that are served by RFC 5322. There is a minor functionality loss to sophisticated users, and there is a real privacy risk (of posting what was intended as a private message, mostly to naive users). Anyway, I don't think this has anything to do with ARC, in the sense that we can assume that we're going to invalidate the original DKIM signature. AFAICT, ARC needs to do its validation on the received message, then add the headers on the way back out. This will work fine in the no-modification case as well. > P.S. Also, I have noticed that there is a feature defined in the > handlers to cleanse all the dkim headers from a mail sent to the > list before it is forwarded to the subscribers. We could simply use > this to extract the previous dkim signature, and generate the ARC > authentication results since its a copy of the previous results. I don't understand what you mean by "extract the previous DKIM signature and generate the ARC authentication results since it's a copy of the previous results." Among other things that seems to assume there was no forward between you and the originating domain that might invalidate the original signature. Now, we need to be very careful to follow the protocol as set out by the RFC. I haven't looked recently (and don't have time to right now), but I seem to recall that each ARC participant must validate all header fields it vouches for (ie, signs in the ARC-Seal). To me that means that we need to provide the DKIM validation service in any case, unless we 100% trust the MTA. That means identifying our MTA's Authentication-Results field, and trusting that our MTA removes any Authentication-Results field that appears to be from our domain, before adding an authentic Authentication-Results. If there are trustworth MTAs in that sense, you could take the strategy of requiring it, but we still might need to do some validation that DKIM-enabled MTAs don't necessarily do, such as earlier ARC fields. Gotta run now, but really before we start trying to design the code, we need to figure out exactly what we're required to do. Steve

1 0