Mailman 3 Search results for "ezmlm" - Mailman-Users

Re: [Mailman-Users] Contemplating new installation

by Brad Knowles

At 7:21 PM +0000 2006-07-26, Jon Loose wrote: > The new project is a fedora core based server, purchased on a subscription. > It runs qmail by default, Ouch. > and has plesk running to give some web-based > admin. Double ouch. > Given the constraint that we need to keep Plesk running, I'd be > most pleased to be able to run mailman on top of qmail. Understood. > First off, I've read that there are some unique issues with qmail/mailman. Yeah, not too many people run with this configuration. Most people using qmail use ezmlm instead of Mailman. That said, I know that at least some versions of Plesk actually integrate a version of Mailman into their system. Unfortunately, what I've seen is a really ancient version (not even the latest 2.0.x release), and depends on an old version of Python, etc.... It is very difficult to upgrade that version of Python and that version of Mailman, without causing everything else on the system to break. I think you need to have a talk with your system provider to see what options are available to you. > Does this make installation significantly harder than with a postfix > system (it was as much as I could do to get the postfix/mailman system > going - but I'm willing to persevere!) What are the main issues? IMO, postfix and sendmail are the most natural MTAs to use with Mailman, although I believe that postfix tends to work better out-of-the-box as an MTA that handles the kinds of loads that a mailing list may tend to generate. You can get equal or better performance from sendmail (especially if you're doing a lot of anti-spam/anti-virus processing or other type of filtering on your mail), but it takes more work to get there with sendmail than it does with postfix, and it takes more work to keep the system operating at that level with sendmail than it does with postfix. But all of this should be covered in the instructions. > Also, > is Plesk likely to foul up the installation if it is used post the > mailman install? Generally speaking, my view is that if you're going to use Plesk, then you should stick with whatever they provide, or at least whatever they make easy to integrate. That may very well mean that Mailman is not a part of that picture, at least not anything we would consider to be a reasonably modern version of Mailman. > Second, my previous install of mailman was 2.1.x running under > /usr/local/mailman. I note that the fedora installation already seems to > have mailman files distributed around the filesystem. Is there a standard > installation now as part of fedora, and am I better to use this or stick > with a fresh installation into /usr/local/mailman? They've done whatever they've done and didn't ask for any involvement from us. You need to choose whether you want to stick with their way of doing things, or if you want to do things our way. If you're going to go with doing things their way, you need to be aware that some of your support issues will need to be addressed to them and not us, and you'll need to be able to figure out which types of support issues need to be addressed where. > One final related issue: this system has a greater need for security than > the last one. Is it simple to administer the web side of mailman using > https so that user login is ecrypted? My personal belief is that all interactions between any user or admin and the web interface of Mailman should go through an SSL encrypted connection. All. That's not too hard to set up, if you follow the instructions. > Is there much needed for apache to > be able to do this? It's not too hard to set up, and apache should be able to handle things just fine. Just make sure to read all the instructions, and to search the FAQ and the archives. -- Brad Knowles, <brad(a)stop.mail-abuse.org> "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See <http://www.lopsa.org/>.

18 years, 5 months

Re: [Mailman-Users] Can this be done using Mailman?

by DAve

Mark Sapiro wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > DAve wrote: >> Brad Knowles wrote: >>> On 8/8/07, DAve wrote: >>> >>>> So in the idea of an announcement only list, we should be able to have >>>> the list owner post a message to multiple topics, correct? I did not see >>>> where cross posting a message to multiple topics within a single list >>>> was possible. Do you know if it is? >>> I would think so, yes. But you should probably wait for an answer from >>> someone who knows better how that part of the code actually works, like >>> maybe Mark Sapiro or Tokio. >>> >> Doesn't look like that answer is coming, I must be treading new ground. >> Is anyone interested in the answer if I setup a test install and try it? > > > Sometimes people are out of electronic contact for weeks at a time :) Hey I wasn't complaining! I figured I would just go try it myself so I did. It works exactly as you describe below which is what we want. Now we just have to figure out how to manage the addresses and user preferences for it looks like it could go 100k subscribers. Currently we use ezmlm and we manage all subscriptions via a management interface and SQL which syncs the subscriber list with the client's AS400. We will check out the xmlrpc patch as well. We are reading the FAQ on high volume lists and searching the archives now. Thanks, DAve > > Topics are defined for a list. > > Each member may subscribe to zero or more of the defined topics, and may > additionally choose to receive messages which don't match any topic. If > the member subscribes to zero topics, she receives all list messages. If > the member subscribes to one or more topics, she receives (one copy) of > all messages matching any of the subscribed topics, and if opted, those > messages which match no topic. > > Each topic definition includes a regexp which is matched against the > Subject: header, the Keywords: header if any, and any initial body lines > that look like Subject: or Keywords: headers up to the first non-header > like body line or topics_bodylines_limit, whichever comes first. > > >> As we understand it you can have topics, and subscribers will only >> receive messages directed at the topics they subscribe to. > > > Plus messages which match no topic if they choose. But, since this is an > announce list, you should be able to guarantee that all messages will > match one or more topics. > > >> This means a >> subscriber could subscribe to [members], [stores], and [retailers], if >> we send a message to [stores] they will receive it. If we send a message >> to all topics, subscribers will receive only one message. This is what >> we want. > > > I don't understand what you mean by "all topics". > > >> However, if we send a message only to [stores] and to [members], >> subscribers will receive two copies. > > > No. Those members subscribed to either [stores] or to [members] or to > both will each receive exactly one copy of the message. > > >> This is not what we want. In order >> to have a subscriber receive only one copy of the message, the message >> must go to ALL topics, [stores], [members], [retailers], [malls], >> [gas_stations], [gas_stations_with_inside_bathrooms], >> [gas_stations_with_outside_bathrooms_in_north_dakota], etc. Not what we >> want. > > > The above is not correct. Each list member will receive either zero or > one message depending on whether or not the message matches at least one > subscribed topic. > > - -- > Mark Sapiro <msapiro(a)value.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.7 (MingW32) > > iD8DBQFGx6sFVVuXXpU7hpMRAqWJAKC/bOquP7vyCeSELnkLDxHM1LQckgCeOHsM > 01pmODUHOCdbIe2DjTCH2Bg= > =WVdO > -----END PGP SIGNATURE----- > > -- Three years now I've asked Google why they don't have a logo change for Memorial Day. Why do they choose to do logos for other non-international holidays, but nothing for Veterans? Maybe they forgot who made that choice possible.

17 years, 4 months

Re: [Mailman-Users] importing archived Maildir email lists into Mailman lists archives

by [IDIS Technical Secretariat] Ricardo Rodríguez

Thanks for your reply! Please, read below! On Tue, Sep 1, 2015 at 3:19 AM, Stephen J. Turnbull <stephen(a)xemacs.org> wrote: > [IDIS Technical Secretariat] Ricardo Rodríguez writes: > > Thanks Mark, all, > > > > On Sun, Aug 30, 2015 at 4:09 AM, Mark Sapiro <mark(a)msapiro.net> wrote: > > > > > On 08/28/2015 12:52 AM, [IDIS Technical Secretariat] Ricardo Rodríguez > > > wrote: > > > > > > > > I have a number of Maildir format mailing lists archives > > No, you don't have Maildir, at least not "Maildir" as most of the > Internet understands it. Here's what Dan Bernstein (the inventor or > at least popularizer of Maildir) says: > > Can a maildir contain more than tmp, new, cur? > > Yes: > .qmail: used to do direct deliveries with qmail-local. > bulletintime: empty file, used by system-wide bulletin programs. > bulletinlock: empty file, used by system-wide bulletin programs. > seriallock: empty file, used to serialize AutoTURN. > > > Ricardo-Rodriguezs-Mac-Pro:r.users rrodriguez$ ls > > Log bounce digissue headerremove lock mod outlocal remote > > "Lock" -- no, this isn't Maildir. The whole point of Maildir is that > you don't need locks because reading and writing are done in different > directories, and changes happen atomically. (This can even work with > editing.) > That's now far clear for me! As stated in a previous message replying to a Mark's post, I make a mess interpreting wrongly several messages from our services provider and some googled information. Ezmlm was behind the scene. Sorry for the misinformation! > > > allow bouncer dignum indexed lockbounce modsub owner subscribers > > archive config editor inlocal mailinglist num prefix text > > archived digest headeradd key manager outhost public tstdig > > > > Within /archive, there are two folders, 0 and 1, with a number of files, > > each of them containing one message, and an index file. > > > > Please, does this made sense for you? > > I don't recall anything like that. Please try to find an explanation > of the structure in the system documentation, or ask the vendor. > However, since you think they're "Maildir", probably what is meant is > that they have a structure that is one message per file rather than > many messages per file. You probably just need to figure out how to > get the order of messages right, then concatenate the messages. > > That's correct: messages are stored in separated files ordered in subfolders named with a series from 0 onward. 0, 1... each subfolder holds one hundred files, each of them with a complete message. I think the global order could be provided by the subfolder order plus the name of each file within a subfolder. Mark's reply contents a simple line to get some concatenation. I'll play with this idea and report back! > Most likely, all you need for each list are the archive folders and > the single messages, and maybe the index file will be of some use > depending on what it contains. If your documentation and/or the old > vendor are of no help, see if you can find a whole message file you > can send to us *as a file attachment* -- we want to see what headers > are included (it probably doesn't really matter, though, except for > the "Unix From_" which they probably don't have). If for privacy > reasons you don't want to broadcast any message on a public mailing > list, you can send it to Mark and me personally. Also it may be > helpful to figure out the rule for the folders whose names are > numbers: are they the leading digits where files are named 000 to 999? > Are they months? Years? etc. > Most lists are public, so privacy is not a concern! You can find here the complete folders' structure for one of the lists... http://datasource.idisantiago.es/r.users/ > > > Am I completely lost? > > No, of course not. Just don't delete anything until you're sure the > new system is working. I did qualify everything with "probably", it > may take a couple of guesses to get it right. :-) > > Thanks for your help! Ricardo -- Ricardo Rodríguez Research Management and Promotion Technician Technical Secretariat Health Research Institute of Santiago de Compostela (IDIS) http://www.idisantiago.es

9 years, 3 months

Re: [Mailman-Users] Contemplating new installation

by Jon Loose

Thanks for the swift replies. Food for thought! No doubt I'll be back to you soon. Jon ----- Original Message ---- From: Brad Knowles <brad(a)stop.mail-abuse.org> To: Jon Loose <jjloose(a)yahoo.co.uk>; mailman-users(a)python.org Sent: Thursday, 27 July, 2006 2:03:29 AM Subject: Re: [Mailman-Users] Contemplating new installation At 7:21 PM +0000 2006-07-26, Jon Loose wrote: > The new project is a fedora core based server, purchased on a subscription. > It runs qmail by default, Ouch. > and has plesk running to give some web-based > admin. Double ouch. > Given the constraint that we need to keep Plesk running, I'd be > most pleased to be able to run mailman on top of qmail. Understood. > First off, I've read that there are some unique issues with qmail/mailman. Yeah, not too many people run with this configuration. Most people using qmail use ezmlm instead of Mailman. That said, I know that at least some versions of Plesk actually integrate a version of Mailman into their system. Unfortunately, what I've seen is a really ancient version (not even the latest 2.0.x release), and depends on an old version of Python, etc.... It is very difficult to upgrade that version of Python and that version of Mailman, without causing everything else on the system to break. I think you need to have a talk with your system provider to see what options are available to you. > Does this make installation significantly harder than with a postfix > system (it was as much as I could do to get the postfix/mailman system > going - but I'm willing to persevere!) What are the main issues? IMO, postfix and sendmail are the most natural MTAs to use with Mailman, although I believe that postfix tends to work better out-of-the-box as an MTA that handles the kinds of loads that a mailing list may tend to generate. You can get equal or better performance from sendmail (especially if you're doing a lot of anti-spam/anti-virus processing or other type of filtering on your mail), but it takes more work to get there with sendmail than it does with postfix, and it takes more work to keep the system operating at that level with sendmail than it does with postfix. But all of this should be covered in the instructions. > Also, > is Plesk likely to foul up the installation if it is used post the > mailman install? Generally speaking, my view is that if you're going to use Plesk, then you should stick with whatever they provide, or at least whatever they make easy to integrate. That may very well mean that Mailman is not a part of that picture, at least not anything we would consider to be a reasonably modern version of Mailman. > Second, my previous install of mailman was 2.1.x running under > /usr/local/mailman. I note that the fedora installation already seems to > have mailman files distributed around the filesystem. Is there a standard > installation now as part of fedora, and am I better to use this or stick > with a fresh installation into /usr/local/mailman? They've done whatever they've done and didn't ask for any involvement from us. You need to choose whether you want to stick with their way of doing things, or if you want to do things our way. If you're going to go with doing things their way, you need to be aware that some of your support issues will need to be addressed to them and not us, and you'll need to be able to figure out which types of support issues need to be addressed where. > One final related issue: this system has a greater need for security than > the last one. Is it simple to administer the web side of mailman using > https so that user login is ecrypted? My personal belief is that all interactions between any user or admin and the web interface of Mailman should go through an SSL encrypted connection. All. That's not too hard to set up, if you follow the instructions. > Is there much needed for apache to > be able to do this? It's not too hard to set up, and apache should be able to handle things just fine. Just make sure to read all the instructions, and to search the FAQ and the archives. -- Brad Knowles, <brad(a)stop.mail-abuse.org> "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See <http://www.lopsa.org/>.

18 years, 5 months

RE: [Mailman-Users] Subscribing to Multiple Lists Simultaneously - SOLVED

by Russell Mann

> Next thing I > want to do is allow the users to subscribe to multiple lists > from one signup page. The reason is we don't want our > customers to have to go through more hoops just to get our > information! > > I see a couple questions on the list posted about this, but > no one appears to have been replied to yet. If this isn't > possible, please at least reply to let me know that! > > If there is a command-list email option, that would work, such as: > > Emaillistname-subscribe(a)host.com where the From info is > parsed to effect the signup. It's easy to write a perl form > to take an inputted email address and send to the subscribe address. Since there seem to be no takers, here's what I'm doing currently. I wrote an HTML form and PERL script to send emails to the <listname>-request(a)host.com address, with "subscribe" in the subject, which seems to do the trick. You can modify the form and script to accommodate any number of email lists. (Superflouous HTML formatting removed) HTML FORM: <form method="post" action="/cgi-bin/email_list_subscribe.pl"> Sign up for list1: <input name="list1" type="checkbox" value="list1name" checked> Sign up for list2: <input name="list2" type="checkbox" value="list2name" checked> Your Name (optional): <input type="text" name="user_name" size="40" maxlength="100"> E-mail Address: <input type="text" name="user_email" size="40" maxlength="100"> <input type="submit" name="doit" value="Subscribe to List1"> <input type="hidden" name="listtarget" value="subscribe"> <input type="hidden" name="acktitle" value="Thank you!<br>Your request has been received."> <input type="hidden" name="acktext" value="Click <a href=/>here</a> to return to the home page."> <input type="hidden" name="counter_file" value="subscribe_counter.txt"> </form> PERL SCRIPT TO HANDLE IT. #!/usr/bin/perl $root_dir = "/www/domain"; #location of qmail-inject on your system $mailer = '/var/qmail/bin/qmail-inject'; # If you're using sendmail, probably change this to /usr/lib/sendmail or something like that. $topfile="$root_dir/templates/top.html"; $bottomfile="$root_dir/templates/bottom.html"; $datapath = "$root_dir/data/list1/"; $counterpath = "$root_dir/data/list1_counters"; $end_message = <<"MESSAGE__"; <p>We have just sent you a message requesting your confirmation that you want to be added to our mailing list. This verifies that we have your correct e-mail address, and protects you from fraudulent subscription requests. Please read this email message and follow the instructions to complete the subscription process.</p> <p>Thank You for subscribing to these email services. If you ever have any questions regarding your subscription, please contact us at <a href="/email/"></a>.</p> MESSAGE__ $unsub_end_message = <<"MESSAGE__"; <p>We have just sent you a message requesting your confirmation that you want to be REMOVED from our mailing list. This verifies that we have your correct e-mail address, and protects you from fraudulent requests. Please read this email message and follow the instructions to complete the unsubscribe process.</p> <p>If you ever have any questions, please contact us at <a href="/email/"></a>.</p> MESSAGE__ # next line should be the domain name of your ezmlm server $maillisthost = 'lists.domain.com'; # list of characters that are OK in an email address $OK_EMAIL_ADDRESS_CHARS='-a-zA-Z0-9_.@'; $OK_NAME_CHARS='-a-zA-Z0-9_. '; # process the user's input read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'}); # Split the name-value pairs @pairs = split(/&/, $buffer); foreach $pair (@pairs) { ($name, $value) = split(/=/, $pair); # Un-Webify plus signs and %-encoding $value =~ tr/+/ /; $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; # Uncomment for debugging purposes # print "Setting $name to $value<P>"; $FORM{$name} = $value; } $addcounter = $FORM{'counter_file'}; $addcounter =~ s/[\.\/]//g; $title = $FORM{'acktitle'}; $top = &Template("$topfile"); $bottom = &Template("$bottomfile"); print "Content-type: text/html\n\n"; if($FORM{list1}) { push @recipients, $FORM{enews}; } if($FORM{list2}) { push @recipients, $FORM{prod}; } # Send subscribe message foreach $myrecipient(@recipients) { $recipient = "$myrecipient-request\@$maillisthost"; $recipient =~ s/[^$OK_EMAIL_ADDRESS_CHARS]/_/go; print ""; $email = $FORM{'user_email'}; $email =~ s/[^$OK_EMAIL_ADDRESS_CHARS]/_/go; $name = $FORM{'user_name'}; $name =~ s/[^$OK_NAME_CHARS]/_/go; print ""; open (MAIL, "|$mailer -f$email $recipient") || die "Can't open $mailer!\n"; print MAIL "From: $name <$email>\n"; print MAIL "Subject: $FORM{listtarget}\n\n"; close (MAIL); } # HTML confirmation notice print "<html><head><title>$title</title>\n"; print "$top"; if ($FORM{listtarget} eq 'subscribe') { print $end_message; } else { print $unsub_end_message; } print "$bottom"; open (FILE, "<$counterpath/$addcounter"); $value = <FILE>; ++$value; close (FILE); open (FILE, ">$counterpath/$addcounter"); print FILE "$value"; close (FILE); sub Template { local(*FILE); # filehandle local($file); # file path local($HTML); # HTML data $file = $_[0] || die "Template : No template file specified\n"; open(FILE, "<$file") || die "template : couldn't open $file : $!\n"; while (<FILE>) { $HTML .= $_; } close(FILE); $HTML =~ s/\$(\w+)/${$1}/g; $HTML =~ s/\$@(\w+)/${$1}/g; return $HTML; } ---------------------------------------------- -- Russell Mann

20 years, 4 months

Re: [Mailman-Users] Having Problems with Different Port

by Seth Lake

so the only advise i can get on this subject from the entire list is 'dude, get a new isp'? that's it? sad. ----- Original Message ----- From: "Seth Lake" <caver(a)verizon.net> To: <mailman-users(a)python.org> Sent: Monday, December 08, 2003 9:46 PM Subject: Re: [Mailman-Users] Having Problems with Different Port > other non-80 port related issues... > > made a group 'Test' locally (via 'www.on-rope.net:8080') and it shows up in > the external listinfo and the internal listinfo, but the other pre-existing > lists still don't show up locally. > > externally if you go to a list info page (1 page in from the entry page) the > bottom link 'Overview of all on-rope.net mailing lists' still manages to > drop the port. > > perhaps there should be a... > DEFAULT_URL_PORT = '80' > ...variable. > > i've put alot of time getting postfix / mailman up and running, it'll be a > shame to have to scrap it all for qmail / ezmlm. > > > ----- Original Message ----- > From: "Seth Lake" <caver(a)verizon.net> > To: <mailman-users(a)python.org> > Sent: Monday, December 08, 2003 3:07 PM > Subject: [Mailman-Users] Having Problems with Different Port > > > Howdy. > > Here's my situation... > > my DSL ISP (verizon) blocks port 80. I registered a domain and found a > dynamic DNS service that will forward requests. i forward 'www.on-rope.net' > to 'on-rope.net:8080'. at my firewall i forward all port 8080 traffic to > port 80 on the www server. so far not bad, but now i try to get mailman > working... > > just setting... > DEFAULT_EMAIL_HOST = 'on-rope.net' > DEFAULT_URL_HOST = 'www.on-rope.net' > ...doesn't work because when your browser hits > 'http://on-rope.net:8080/mailman/listinfo' mailman says 'no lists on > on-rope.net:8080'. okay, so try and set... > DEFAULT_EMAIL_HOST = 'on-rope.net' > DEFAULT_URL_HOST = 'on-rope.net:8080' > ...doesn't work right. on the main index page there's a main email address, > 'mailman(a)on-rope.net'. but instead of that, what it should be, it says > 'mailman@on-rope.net:8080' and i can't figure out where this is set or how > to change it. > > another problem i have is when a user clicks on the archive link the link > tries to send the user to 'http://www.on-rope.net/mailman/private/<list>/'. > interestingly, this does not happen on empty archives, only after a message > has gone though the list. i fixed / hacked this problem by putting this in > my config file... > PUBLIC_ARCHIVE_URL = 'http://on-rope.net:8080/pipermail/%(listname)s' > ...which seemed to work. > > so okay, now i seem to have pretty much everything working okay except for > the mailman email address on the main page. no big deal, i guess i can live > with that. i'm doing all this at work so i can make sure it works from > outside the firewall and ISP's blocked ports and all that. and, at the end > of the day, i go home. > > hmm, more problems. i can access any of my web stuff at home. when i go to > my web server ('http://www.on-rope.net/mailman/listinfo' from home, no need > for port forwarding) there are no lists. i edit my apache config and add > 'Listen *:8080' and then try 'www.on-rope.net:8080' but mailman still says > no lists. this may be because of the 'www' but without the hostname my > brwser just spins it's wheels at home, doesn't know where to get the page > from. i notice that there's no port in the page heading when i'm at home, > at work the main heading is 'on-rope.net:8080 Mailing Lists' but when i'm at > home the heading is 'www.on-rope.net Mailing Lists' no matter what port i > put in. > > so how can i access the same lists from home and abroad? i don't need/want > to host more than a few lists and i have a feeling that the virtual host > system is what's messing with me. is there a way to turn it off? i tried > this... > add_virtualhost('www.on-rope.net', DEFAULT_EMAIL_HOST) > ...to no avail. any help is greatly appreciated. > ------------------------------------------------------ > Mailman-Users mailing list > Mailman-Users(a)python.org > http://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > > This message was sent to: caver(a)verizon.net > Unsubscribe or change your options at > http://mail.python.org/mailman/options/mailman-users/caver%40verizon.net > > > > ------------------------------------------------------ > Mailman-Users mailing list > Mailman-Users(a)python.org > http://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > > This message was sent to: caver(a)verizon.net > Unsubscribe or change your options at > http://mail.python.org/mailman/options/mailman-users/caver%40verizon.net

21 years

Several big changes slated for 2.1.5 (long)

by Barry Warsaw

I have some rather big changes ready for MM2.1.5 that I wanted to describe and get feedback from you. While I have this stuff working and ready to be checked in, we will definitely need some beta testing before unleashing on the world. I hope you'll be able to help with that. I think these changes are important enough to put into 2.1 rather than waiting for any future major release. The first big change is the most externally visible one. I believe the current scheme for bounce processing is unusable in today's world of MyDoom sender forgeries, anti-virus front-ends on remote SMTPs and the like. On python.org we've seen many cases where people are getting unexpectedly bounce disabled, even though they receive all legitimate traffic to a mailing list. What's happening? It's simply that, while we have spam and virus defenses in place on python.org, some crap still gets through. Imagine I'm on a busy list and I forward barry(a)python.org through my home ISP, which has a virus and spam detector on that address. Now say that list gets 100 msgs/day and 1% of those messages are false negative spams. The message gets onto the list, but my ISP catches them and rejects them, which triggers a bounce and thus my score's just been increased by 1. I only need one sneaky spam per day to get me bounce disabled, even though most of the mail is legit and gets through. So I've implemented a revised scheme that we've talked about before, based on what I believe ezmlm does. All the bounce parameters are still in effect, however when a member's bounce score reaches the threshold, we now send a specially prepared probe message containing a VERP'd sender with an unguessable token. When we send the probe, the member's bounce score is reset. If the probe bounces, then we disable the member and do the normal reminders. If the probe doesn't bounce the member would stay enabled and their score starts accumulating from zero again. A benefit of this rewrite is that we can include in the probe, the last bouncing message as a sample to the user so they can start to get a clue as to why they're getting bounce scored. This change has prompted an internal rewrite of the pending database. Previously the entire site had a single pending.pck file for all actions requiring confirmation by the user -- held subscription cancellation, subscription, unsub, and change of address confirmations, and bounce re-enable confirmations. This was a problem for several reasons, including that every list had to block on acquiring the lock for this file. Now, each list has its own pending.pck file and while the list lock must be acquired to update this database, at least this doesn't block other lists from doing things. The upgrade script attempts to migrate the single shared pending.pck file to the individual list files, but the conversion is difficult because the associated list is not stored with most of the records in that file. I do my best, but it's possible that some pending actions may get lost. The other big change is a purely internal one, but it may affect the work flow for some admins. I've changed the qfiles file format so that only one file is used per message. Previously we had one file for the message and one file for the metadata. Now, a single pickle file is used with the first object in the pickle being the message object and the second being the metadata dictionary. This approach has several advantages. The code is simpler, there are no race conditions opportunities, we can't possibly have orphaned data files, and probably most importantly we now only need half the inodes we did before. In addition, I've decided to turn on fsync'ing for this new qfile all the time, so storage should be more reliable too. The downside is that I've removed the ability to set a METADATA_FORMAT. We use Python pickles and that's it. I doubt many people have been using (or were even aware of) the alternatives, although I've had the occasional bug report on them so I know that number is non-zero. The other downside for some people is that the behavior of SAVE_MSGS_AS_PICKLES=False will change. When that non-standard setting is used, we'll still write everything to a pickle file, but we'll use text pickles instead of the more efficient (but not human readable) binary pickles. Also, we'll write the message object as a pickled string instead of a pickled object. Again, this will be less efficient because we'll have to parse the message every time it's dequeued, but this option will still allow people to edit queued messages with a normal text editor, albeit less conveniently. I think this trade-off is worth it. The upgrade script will combine any existing qfiles so you won't have to clear your queue when upgrading. To be safe, you /will/ have to stop Mailman, your MTA, and your web server before upgrading (but this was always recommended practice). I intend to commit these changes to CVS within a week and will probably release a 2.1.5 alpha. This will touch a lot of files, but it will hopefully make the system more efficient and usable. Once this is done I hope to have more time to start addressing other bugs and issues in the 2.1 branch. Again, when everything's checked it, please test things out as much as possible, especially if you are using older Python versions. I've tested primarily with Python 2.3.3 but I was careful not to use any feature that isn't supported in Python 2.1.3. I might have missed something though. -Barry

20 years, 10 months

Re: [Mailman-Users] Can we use an external mail server with mailman ?

by Lloyd F. Tennison

Actually, I found a mod someone had written to do exactly that. In mm_cfg.py need to set these values: SMTPHOST = 'mail.newserver' #SMTP_USERNAME = 'username' #SMTP_PASSWORD = 'password' #DELIVERY_MODULE = 'SMTPDirect' DELIVERY_MODULE = 'ASMTPDirect' # Needs to be 1 for ASMTPDirect to work SMTP_AUTH = 1 and use the attached ASMTPDirect file. I no longer use this, so this has not been updated to 2.16, so you mau want to check it all out. The was orginally received from someone else on this list, so I do not take credit for it. From: John Dennis <jdennis(a)redhat.com> To: julien grosjean <j.grosjean(a)proxiad.com> Date sent: Fri, 23 Sep 2005 10:53:12 -0400 Copies to: mailman-users(a)python.org Subject: Re: [Mailman-Users] Can we use an external mail server with mailman ? > On Fri, 2005-09-23 at 16:27 +0200, julien grosjean wrote: > > MAIL SERVER DETAILS > > > > My Mail Server is NOT on the same server than my mailman server... > > My mail server is proxiad.com (217.109.179.10) with smtp.proxiad.com and > > pop.proxiad.com (This server requiert an authentification... ) > > > > My mailman server is extranet-proxiad.com (193.252.219.206) > > > > All emails are create on the mail server. > > The mail server receive the mail correctly. > > With my mail client, test(a)proxiad.com receive the mail, but not > > suscribers... > > > THE PROBLEM : > > > > I create a test list via web interface. > > > > We can suscribe to the list, but the problem is than when we sent an > > email to the list email, such as test(a)proxiad.com, > > just this email receive the mail... > > Only test(a)proxiad.com receive the mail from the sender... > > Is there a configuration to make ? > > > > > > > > Perhaps there is an aliases problem ? > > > > Should aliases be on the mail server or on the mailman server ? > > Part of the problem is the term mail server can be confusing because > there are two mail servers in the picture, a receiving mail server and a > sending mail server. Mailman needs run on the same machine the receiving > mail server is on otherwise your aliases will not work. The reason is > because when the receving mail server receives an email destined for a > mailman list the mail server will via its aliases determine the > destination for that list email is a program called mailman, thats what > all those aliases are all about. The receiving mail server then hands > the incoming list email to the program mailman. It can only give it to > the program mailman if mailman is running on the same machine. > > Mailman in turn takes the email it received and looks up all the members > of the list and forwards the email to each of them. To do this mailman > invokes a sending mail server. Often the sending and receiving mail > servers (SMTP) are the same and run on the same box. But you can direct > mailman to use a different sending SMTP server via the SMTPHOST > variable. > > Summary: mailman and the receiving SMTP server must be on the same > machine. > > You description is a bit confusing because the way I read it mailman has > in fact received the list email (your previous post shows mailman > sending smtp posts). It could only have done that if it the its running > on the same machine as the receiving SMTP server and the aliaes are > correct. I suspect what actually occurred is those are examples from > mail sent on the machine mailman is running on where everything will > resolve correctly. > -- > John Dennis <jdennis(a)redhat.com> > > ------------------------------------------------------ > Mailman-Users mailing list > Mailman-Users(a)python.org > http://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-users/lists%40viplist.us > > Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp > > # Copyright (C) 1998-2003 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. """Local SMTP direct drop-off w/Authtenication. This module delivers messages via SMTP to a locally specified daemon. This should be compatible with any modern SMTP server. It is expected that the MTA handles all final delivery. We have to play tricks so that the list object isn't locked while delivery occurs synchronously. Note: This file only handles single threaded delivery. See SMTPThreaded.py for a threaded implementation. """ import copy import time import socket import smtplib from types import UnicodeType from Mailman import mm_cfg from Mailman import Utils from Mailman import Errors from Mailman.Handlers import Decorate from Mailman.Logging.Syslog import syslog from Mailman.SafeDict import MsgSafeDict import email from email.Utils import formataddr from email.Header import Header from email.Charset import Charset DOT = '.' # Manage a connection to the SMTP server class Connection: def __init__(self): self.__connect() def __connect(self): self.__conn = smtplib.SMTP() self.__conn.connect(mm_cfg.SMTPHOST, mm_cfg.SMTPPORT) if mm_cfg.SMTP_AUTH: self.__conn.login(mm_cfg.SMTP_USERNAME, mm_cfg.SMTP_PASSWORD) self.__numsessions = mm_cfg.SMTP_MAX_SESSIONS_PER_CONNECTION def sendmail(self, envsender, recips, msgtext): try: results = self.__conn.sendmail(envsender, recips, msgtext) except smtplib.SMTPException: # For safety, reconnect self.__conn.quit() self.__connect() # Let exceptions percolate up raise # Decrement the session counter, reconnecting if necessary self.__numsessions -= 1 # By testing exactly for equality to 0, we automatically handle the # case for SMTP_MAX_SESSIONS_PER_CONNECTION <= 0 meaning never close # the connection. We won't worry about wraparound <wink>. if self.__numsessions == 0: self.__conn.quit() self.__connect() return results def quit(self): self.__conn.quit() def process(mlist, msg, msgdata): recips = msgdata.get('recips') if not recips: # Nobody to deliver to! return # Calculate the non-VERP envelope sender. envsender = msgdata.get('envsender') if envsender is None: if mlist: envsender = mlist.GetBouncesEmail() else: envsender = Utils.get_site_email(extra='bounces') # Time to split up the recipient list. If we're personalizing or VERPing # then each chunk will have exactly one recipient. We'll then hand craft # an envelope sender and stitch a message together in memory for each one # separately. If we're not VERPing, then we'll chunkify based on # SMTP_MAX_RCPTS. Note that most MTAs have a limit on the number of # recipients they'll swallow in a single transaction. deliveryfunc = None if (not msgdata.has_key('personalize') or msgdata['personalize']) and ( msgdata.get('verp') or mlist.personalize): chunks = [[recip] for recip in recips] msgdata['personalize'] = 1 deliveryfunc = verpdeliver elif mm_cfg.SMTP_MAX_RCPTS <= 0: chunks = [recips] else: chunks = chunkify(recips, mm_cfg.SMTP_MAX_RCPTS) # See if this is an unshunted message for which some were undelivered if msgdata.has_key('undelivered'): chunks = msgdata['undelivered'] # If we're doing bulk delivery, then we can stitch up the message now. if deliveryfunc is None: # Be sure never to decorate the message more than once! if not msgdata.get('decorated'): Decorate.process(mlist, msg, msgdata) msgdata['decorated'] = 1 deliveryfunc = bulkdeliver refused = {} t0 = time.time() # Open the initial connection origrecips = msgdata['recips'] # `undelivered' is a copy of chunks that we pop from to do deliveries. # This seems like a good tradeoff between robustness and resource # utilization. If delivery really fails (i.e. qfiles/shunt type # failures), then we'll pick up where we left off with `undelivered'. # This means at worst, the last chunk for which delivery was attempted # could get duplicates but not every one, and no recips should miss the # message. conn = Connection() try: msgdata['undelivered'] = chunks while chunks: chunk = chunks.pop() msgdata['recips'] = chunk try: deliveryfunc(mlist, msg, msgdata, envsender, refused, conn) except Exception: # If /anything/ goes wrong, push the last chunk back on the # undelivered list and re-raise the exception. We don't know # how many of the last chunk might receive the message, so at # worst, everyone in this chunk will get a duplicate. Sigh. chunks.append(chunk) raise del msgdata['undelivered'] finally: conn.quit() msgdata['recips'] = origrecips # Log the successful post t1 = time.time() d = MsgSafeDict(msg, {'time' : t1-t0, # BAW: Urg. This seems inefficient. 'size' : len(msg.as_string()), '#recips' : len(recips), '#refused': len(refused), 'listname': mlist.internal_name(), 'sender' : msg.get_sender(), }) # We have to use the copy() method because extended call syntax requires a # concrete dictionary object; it does not allow a generic mapping. It's # still worthwhile doing the interpolation in syslog() because it'll catch # any catastrophic exceptions due to bogus format strings. if mm_cfg.SMTP_LOG_EVERY_MESSAGE: syslog.write_ex(mm_cfg.SMTP_LOG_EVERY_MESSAGE[0], mm_cfg.SMTP_LOG_EVERY_MESSAGE[1], kws=d) if refused: if mm_cfg.SMTP_LOG_REFUSED: syslog.write_ex(mm_cfg.SMTP_LOG_REFUSED[0], mm_cfg.SMTP_LOG_REFUSED[1], kws=d) elif msgdata.get('tolist'): # Log the successful post, but only if it really was a post to the # mailing list. Don't log sends to the -owner, or -admin addrs. # -request addrs should never get here. BAW: it may be useful to log # the other messages, but in that case, we should probably have a # separate configuration variable to control that. if mm_cfg.SMTP_LOG_SUCCESS: syslog.write_ex(mm_cfg.SMTP_LOG_SUCCESS[0], mm_cfg.SMTP_LOG_SUCCESS[1], kws=d) # Process any failed deliveries. tempfailures = [] permfailures = [] for recip, (code, smtpmsg) in refused.items(): # DRUMS is an internet draft, but it says: # # [RFC-821] incorrectly listed the error where an SMTP server # exhausts its implementation limit on the number of RCPT commands # ("too many recipients") as having reply code 552. The correct # reply code for this condition is 452. Clients SHOULD treat a 552 # code in this case as a temporary, rather than permanent failure # so the logic below works. # if code >= 500 and code <> 552: # A permanent failure permfailures.append(recip) else: # Deal with persistent transient failures by queuing them up for # future delivery. TBD: this could generate lots of log entries! tempfailures.append(recip) if mm_cfg.SMTP_LOG_EACH_FAILURE: d.update({'recipient': recip, 'failcode' : code, 'failmsg' : smtpmsg}) syslog.write_ex(mm_cfg.SMTP_LOG_EACH_FAILURE[0], mm_cfg.SMTP_LOG_EACH_FAILURE[1], kws=d) # Return the results if tempfailures or permfailures: raise Errors.SomeRecipientsFailed(tempfailures, permfailures) def chunkify(recips, chunksize): # First do a simple sort on top level domain. It probably doesn't buy us # much to try to sort on MX record -- that's the MTA's job. We're just # trying to avoid getting a max recips error. Split the chunks along # these lines (as suggested originally by Chuq Von Rospach and slightly # elaborated by BAW). chunkmap = {'com': 1, 'net': 2, 'org': 2, 'edu': 3, 'us' : 3, 'ca' : 3, } buckets = {} for r in recips: tld = None i = r.rfind('.') if i >= 0: tld = r[i+1:] bin = chunkmap.get(tld, 0) bucket = buckets.get(bin, []) bucket.append(r) buckets[bin] = bucket # Now start filling the chunks chunks = [] currentchunk = [] chunklen = 0 for bin in buckets.values(): for r in bin: currentchunk.append(r) chunklen = chunklen + 1 if chunklen >= chunksize: chunks.append(currentchunk) currentchunk = [] chunklen = 0 if currentchunk: chunks.append(currentchunk) currentchunk = [] chunklen = 0 return chunks def verpdeliver(mlist, msg, msgdata, envsender, failures, conn): for recip in msgdata['recips']: # We now need to stitch together the message with its header and # footer. If we're VERPIng, we have to calculate the envelope sender # for each recipient. Note that the list of recipients must be of # length 1. # # BAW: ezmlm includes the message number in the envelope, used when # sending a notification to the user telling her how many messages # they missed due to bouncing. Neat idea. msgdata['recips'] = [recip] # Make a copy of the message and decorate + delivery that msgcopy = copy.deepcopy(msg) Decorate.process(mlist, msgcopy, msgdata) # Calculate the envelope sender, which we may be VERPing if msgdata.get('verp'): bmailbox, bdomain = Utils.ParseEmail(envsender) rmailbox, rdomain = Utils.ParseEmail(recip) if rdomain is None: # The recipient address is not fully-qualified. We can't # deliver it to this person, nor can we craft a valid verp # header. I don't think there's much we can do except ignore # this recipient. syslog('smtp', 'Skipping VERP delivery to unqual recip: %s', recip) continue d = {'bounces': bmailbox, 'mailbox': rmailbox, 'host' : DOT.join(rdomain), } envsender = '%s@%s' % ((mm_cfg.VERP_FORMAT % d), DOT.join(bdomain)) if mlist.personalize == 2: # When fully personalizing, we want the To address to point to the # recipient, not to the mailing list del msgcopy['to'] name = None if mlist.isMember(recip): name = mlist.getMemberName(recip) if name: # Convert the name to an email-safe representation. If the # name is a byte string, convert it first to Unicode, given # the character set of the member's language, replacing bad # characters for which we can do nothing about. Once we have # the name as Unicode, we can create a Header instance for it # so that it's properly encoded for email transport. charset = Utils.GetCharSet(mlist.getMemberLanguage(recip)) if charset == 'us-ascii': # Since Header already tries both us-ascii and utf-8, # let's add something a bit more useful. charset = 'iso-8859-1' charset = Charset(charset) codec = charset.input_codec or 'ascii' if not isinstance(name, UnicodeType): name = unicode(name, codec, 'replace') name = Header(name, charset).encode() msgcopy['To'] = formataddr((name, recip)) else: msgcopy['To'] = recip # We can flag the mail as a duplicate for each member, if they've # already received this message, as calculated by Message-ID. See # AvoidDuplicates.py for details. del msgcopy['x-mailman-copy'] if msgdata.get('add-dup-header', {}).has_key(recip): msgcopy['X-Mailman-Copy'] = 'yes' # For the final delivery stage, we can just bulk deliver to a party of # one. ;) bulkdeliver(mlist, msgcopy, msgdata, envsender, failures, conn) def bulkdeliver(mlist, msg, msgdata, envsender, failures, conn): # Do some final cleanup of the message header. Start by blowing away # any the Sender: and Errors-To: headers so remote MTAs won't be # tempted to delivery bounces there instead of our envelope sender del msg['sender'] del msg['errors-to'] msg['Sender'] = envsender msg['Errors-To'] = envsender # Get the plain, flattened text of the message, sans unixfrom msgtext = msg.as_string() refused = {} recips = msgdata['recips'] try: # Send the message refused = conn.sendmail(envsender, recips, msgtext) except smtplib.SMTPRecipientsRefused, e: refused = e.recipients # MTA not responding, or other socket problems, or any other kind of # SMTPException. In that case, nothing got delivered except (socket.error, smtplib.SMTPException, IOError), e: # BAW: should this be configurable? syslog('smtp', 'All recipients refused: %s', e) # If the exception had an associated error code, use it, otherwise, # fake it with a non-triggering exception code errcode = getattr(e, 'smtp_code', -1) errmsg = getattr(e, 'smtp_error', 'ignore') for r in recips: refused[r] = (errcode, errmsg) failures.update(refused)

19 years, 3 months

Re: [Mailman-Users] Can we use an external mail server with mailman ?

by game lover

This is a topic I am interested in. But can someone summarize these codes to make them simple so most people can following them easily? ----------------------------------------------------------------- http://www.FreeGame123.com All the games are free to play online. Playing is the best way of learning. "Lloyd F. Tennison" <lists(a)viplist.us> wrote:Actually, I found a mod someone had written to do exactly that. In mm_cfg.py need to set these values: SMTPHOST = 'mail.newserver' #SMTP_USERNAME = 'username' #SMTP_PASSWORD = 'password' #DELIVERY_MODULE = 'SMTPDirect' DELIVERY_MODULE = 'ASMTPDirect' # Needs to be 1 for ASMTPDirect to work SMTP_AUTH = 1 and use the attached ASMTPDirect file. I no longer use this, so this has not been updated to 2.16, so you mau want to check it all out. The was orginally received from someone else on this list, so I do not take credit for it. From: John Dennis To: julien grosjean Date sent: Fri, 23 Sep 2005 10:53:12 -0400 Copies to: mailman-users(a)python.org Subject: Re: [Mailman-Users] Can we use an external mail server with mailman ? > On Fri, 2005-09-23 at 16:27 +0200, julien grosjean wrote: > > MAIL SERVER DETAILS > > > > My Mail Server is NOT on the same server than my mailman server... > > My mail server is proxiad.com (217.109.179.10) with smtp.proxiad.com and > > pop.proxiad.com (This server requiert an authentification... ) > > > > My mailman server is extranet-proxiad.com (193.252.219.206) > > > > All emails are create on the mail server. > > The mail server receive the mail correctly. > > With my mail client, test(a)proxiad.com receive the mail, but not > > suscribers... > > > THE PROBLEM : > > > > I create a test list via web interface. > > > > We can suscribe to the list, but the problem is than when we sent an > > email to the list email, such as test(a)proxiad.com, > > just this email receive the mail... > > Only test(a)proxiad.com receive the mail from the sender... > > Is there a configuration to make ? > > > > > > > > Perhaps there is an aliases problem ? > > > > Should aliases be on the mail server or on the mailman server ? > > Part of the problem is the term mail server can be confusing because > there are two mail servers in the picture, a receiving mail server and a > sending mail server. Mailman needs run on the same machine the receiving > mail server is on otherwise your aliases will not work. The reason is > because when the receving mail server receives an email destined for a > mailman list the mail server will via its aliases determine the > destination for that list email is a program called mailman, thats what > all those aliases are all about. The receiving mail server then hands > the incoming list email to the program mailman. It can only give it to > the program mailman if mailman is running on the same machine. > > Mailman in turn takes the email it received and looks up all the members > of the list and forwards the email to each of them. To do this mailman > invokes a sending mail server. Often the sending and receiving mail > servers (SMTP) are the same and run on the same box. But you can direct > mailman to use a different sending SMTP server via the SMTPHOST > variable. > > Summary: mailman and the receiving SMTP server must be on the same > machine. > > You description is a bit confusing because the way I read it mailman has > in fact received the list email (your previous post shows mailman > sending smtp posts). It could only have done that if it the its running > on the same machine as the receiving SMTP server and the aliaes are > correct. I suspect what actually occurred is those are examples from > mail sent on the machine mailman is running on where everything will > resolve correctly. > -- > John Dennis > > ------------------------------------------------------ > Mailman-Users mailing list > Mailman-Users(a)python.org > http://mail.python.org/mailman/listinfo/mailman-users > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-users/lists%40viplist.us > > Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp > > # Copyright (C) 1998-2003 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. """Local SMTP direct drop-off w/Authtenication. This module delivers messages via SMTP to a locally specified daemon. This should be compatible with any modern SMTP server. It is expected that the MTA handles all final delivery. We have to play tricks so that the list object isn't locked while delivery occurs synchronously. Note: This file only handles single threaded delivery. See SMTPThreaded.py for a threaded implementation. """ import copy import time import socket import smtplib from types import UnicodeType from Mailman import mm_cfg from Mailman import Utils from Mailman import Errors from Mailman.Handlers import Decorate from Mailman.Logging.Syslog import syslog from Mailman.SafeDict import MsgSafeDict import email from email.Utils import formataddr from email.Header import Header from email.Charset import Charset DOT = '.' # Manage a connection to the SMTP server class Connection: def __init__(self): self.__connect() def __connect(self): self.__conn = smtplib.SMTP() self.__conn.connect(mm_cfg.SMTPHOST, mm_cfg.SMTPPORT) if mm_cfg.SMTP_AUTH: self.__conn.login(mm_cfg.SMTP_USERNAME, mm_cfg.SMTP_PASSWORD) self.__numsessions = mm_cfg.SMTP_MAX_SESSIONS_PER_CONNECTION def sendmail(self, envsender, recips, msgtext): try: results = self.__conn.sendmail(envsender, recips, msgtext) except smtplib.SMTPException: # For safety, reconnect self.__conn.quit() self.__connect() # Let exceptions percolate up raise # Decrement the session counter, reconnecting if necessary self.__numsessions -= 1 # By testing exactly for equality to 0, we automatically handle the # case for SMTP_MAX_SESSIONS_PER_CONNECTION <= 0 meaning never close # the connection. We won't worry about wraparound . if self.__numsessions == 0: self.__conn.quit() self.__connect() return results def quit(self): self.__conn.quit() def process(mlist, msg, msgdata): recips = msgdata.get('recips') if not recips: # Nobody to deliver to! return # Calculate the non-VERP envelope sender. envsender = msgdata.get('envsender') if envsender is None: if mlist: envsender = mlist.GetBouncesEmail() else: envsender = Utils.get_site_email(extra='bounces') # Time to split up the recipient list. If we're personalizing or VERPing # then each chunk will have exactly one recipient. We'll then hand craft # an envelope sender and stitch a message together in memory for each one # separately. If we're not VERPing, then we'll chunkify based on # SMTP_MAX_RCPTS. Note that most MTAs have a limit on the number of # recipients they'll swallow in a single transaction. deliveryfunc = None if (not msgdata.has_key('personalize') or msgdata['personalize']) and ( msgdata.get('verp') or mlist.personalize): chunks = [[recip] for recip in recips] msgdata['personalize'] = 1 deliveryfunc = verpdeliver elif mm_cfg.SMTP_MAX_RCPTS <= 0: chunks = [recips] else: chunks = chunkify(recips, mm_cfg.SMTP_MAX_RCPTS) # See if this is an unshunted message for which some were undelivered if msgdata.has_key('undelivered'): chunks = msgdata['undelivered'] # If we're doing bulk delivery, then we can stitch up the message now. if deliveryfunc is None: # Be sure never to decorate the message more than once! if not msgdata.get('decorated'): Decorate.process(mlist, msg, msgdata) msgdata['decorated'] = 1 deliveryfunc = bulkdeliver refused = {} t0 = time.time() # Open the initial connection origrecips = msgdata['recips'] # `undelivered' is a copy of chunks that we pop from to do deliveries. # This seems like a good tradeoff between robustness and resource # utilization. If delivery really fails (i.e. qfiles/shunt type # failures), then we'll pick up where we left off with `undelivered'. # This means at worst, the last chunk for which delivery was attempted # could get duplicates but not every one, and no recips should miss the # message. conn = Connection() try: msgdata['undelivered'] = chunks while chunks: chunk = chunks.pop() msgdata['recips'] = chunk try: deliveryfunc(mlist, msg, msgdata, envsender, refused, conn) except Exception: # If /anything/ goes wrong, push the last chunk back on the # undelivered list and re-raise the exception. We don't know # how many of the last chunk might receive the message, so at # worst, everyone in this chunk will get a duplicate. Sigh. chunks.append(chunk) raise del msgdata['undelivered'] finally: conn.quit() msgdata['recips'] = origrecips # Log the successful post t1 = time.time() d = MsgSafeDict(msg, {'time' : t1-t0, # BAW: Urg. This seems inefficient. 'size' : len(msg.as_string()), '#recips' : len(recips), '#refused': len(refused), 'listname': mlist.internal_name(), 'sender' : msg.get_sender(), }) # We have to use the copy() method because extended call syntax requires a # concrete dictionary object; it does not allow a generic mapping. It's # still worthwhile doing the interpolation in syslog() because it'll catch # any catastrophic exceptions due to bogus format strings. if mm_cfg.SMTP_LOG_EVERY_MESSAGE: syslog.write_ex(mm_cfg.SMTP_LOG_EVERY_MESSAGE[0], mm_cfg.SMTP_LOG_EVERY_MESSAGE[1], kws=d) if refused: if mm_cfg.SMTP_LOG_REFUSED: syslog.write_ex(mm_cfg.SMTP_LOG_REFUSED[0], mm_cfg.SMTP_LOG_REFUSED[1], kws=d) elif msgdata.get('tolist'): # Log the successful post, but only if it really was a post to the # mailing list. Don't log sends to the -owner, or -admin addrs. # -request addrs should never get here. BAW: it may be useful to log # the other messages, but in that case, we should probably have a # separate configuration variable to control that. if mm_cfg.SMTP_LOG_SUCCESS: syslog.write_ex(mm_cfg.SMTP_LOG_SUCCESS[0], mm_cfg.SMTP_LOG_SUCCESS[1], kws=d) # Process any failed deliveries. tempfailures = [] permfailures = [] for recip, (code, smtpmsg) in refused.items(): # DRUMS is an internet draft, but it says: # # [RFC-821] incorrectly listed the error where an SMTP server # exhausts its implementation limit on the number of RCPT commands # ("too many recipients") as having reply code 552. The correct # reply code for this condition is 452. Clients SHOULD treat a 552 # code in this case as a temporary, rather than permanent failure # so the logic below works. # if code >= 500 and code <> 552: # A permanent failure permfailures.append(recip) else: # Deal with persistent transient failures by queuing them up for # future delivery. TBD: this could generate lots of log entries! tempfailures.append(recip) if mm_cfg.SMTP_LOG_EACH_FAILURE: d.update({'recipient': recip, 'failcode' : code, 'failmsg' : smtpmsg}) syslog.write_ex(mm_cfg.SMTP_LOG_EACH_FAILURE[0], mm_cfg.SMTP_LOG_EACH_FAILURE[1], kws=d) # Return the results if tempfailures or permfailures: raise Errors.SomeRecipientsFailed(tempfailures, permfailures) def chunkify(recips, chunksize): # First do a simple sort on top level domain. It probably doesn't buy us # much to try to sort on MX record -- that's the MTA's job. We're just # trying to avoid getting a max recips error. Split the chunks along # these lines (as suggested originally by Chuq Von Rospach and slightly # elaborated by BAW). chunkmap = {'com': 1, 'net': 2, 'org': 2, 'edu': 3, 'us' : 3, 'ca' : 3, } buckets = {} for r in recips: tld = None i = r.rfind('.') if i >= 0: tld = r[i+1:] bin = chunkmap.get(tld, 0) bucket = buckets.get(bin, []) bucket.append(r) buckets[bin] = bucket # Now start filling the chunks chunks = [] currentchunk = [] chunklen = 0 for bin in buckets.values(): for r in bin: currentchunk.append(r) chunklen = chunklen + 1 if chunklen >= chunksize: chunks.append(currentchunk) currentchunk = [] chunklen = 0 if currentchunk: chunks.append(currentchunk) currentchunk = [] chunklen = 0 return chunks def verpdeliver(mlist, msg, msgdata, envsender, failures, conn): for recip in msgdata['recips']: # We now need to stitch together the message with its header and # footer. If we're VERPIng, we have to calculate the envelope sender # for each recipient. Note that the list of recipients must be of # length 1. # # BAW: ezmlm includes the message number in the envelope, used when # sending a notification to the user telling her how many messages # they missed due to bouncing. Neat idea. msgdata['recips'] = [recip] # Make a copy of the message and decorate + delivery that msgcopy = copy.deepcopy(msg) Decorate.process(mlist, msgcopy, msgdata) # Calculate the envelope sender, which we may be VERPing if msgdata.get('verp'): bmailbox, bdomain = Utils.ParseEmail(envsender) rmailbox, rdomain = Utils.ParseEmail(recip) if rdomain is None: # The recipient address is not fully-qualified. We can't # deliver it to this person, nor can we craft a valid verp # header. I don't think there's much we can do except ignore # this recipient. syslog('smtp', 'Skipping VERP delivery to unqual recip: %s', recip) continue d = {'bounces': bmailbox, 'mailbox': rmailbox, 'host' : DOT.join(rdomain), } envsender = '%s@%s' % ((mm_cfg.VERP_FORMAT % d), DOT.join(bdomain)) if mlist.personalize == 2: # When fully personalizing, we want the To address to point to the # recipient, not to the mailing list del msgcopy['to'] name = None if mlist.isMember(recip): name = mlist.getMemberName(recip) if name: # Convert the name to an email-safe representation. If the # name is a byte string, convert it first to Unicode, given # the character set of the member's language, replacing bad # characters for which we can do nothing about. Once we have # the name as Unicode, we can create a Header instance for it # so that it's properly encoded for email transport. charset = Utils.GetCharSet(mlist.getMemberLanguage(recip)) if charset == 'us-ascii': # Since Header already tries both us-ascii and utf-8, # let's add something a bit more useful. charset = 'iso-8859-1' charset = Charset(charset) codec = charset.input_codec or 'ascii' if not isinstance(name, UnicodeType): name = unicode(name, codec, 'replace') name = Header(name, charset).encode() msgcopy['To'] = formataddr((name, recip)) else: msgcopy['To'] = recip # We can flag the mail as a duplicate for each member, if they've # already received this message, as calculated by Message-ID. See # AvoidDuplicates.py for details. del msgcopy['x-mailman-copy'] if msgdata.get('add-dup-header', {}).has_key(recip): msgcopy['X-Mailman-Copy'] = 'yes' # For the final delivery stage, we can just bulk deliver to a party of # one. ;) bulkdeliver(mlist, msgcopy, msgdata, envsender, failures, conn) def bulkdeliver(mlist, msg, msgdata, envsender, failures, conn): # Do some final cleanup of the message header. Start by blowing away # any the Sender: and Errors-To: headers so remote MTAs won't be # tempted to delivery bounces there instead of our envelope sender del msg['sender'] del msg['errors-to'] msg['Sender'] = envsender msg['Errors-To'] = envsender # Get the plain, flattened text of the message, sans unixfrom msgtext = msg.as_string() refused = {} recips = msgdata['recips'] try: # Send the message refused = conn.sendmail(envsender, recips, msgtext) except smtplib.SMTPRecipientsRefused, e: refused = e.recipients # MTA not responding, or other socket problems, or any other kind of # SMTPException. In that case, nothing got delivered except (socket.error, smtplib.SMTPException, IOError), e: # BAW: should this be configurable? syslog('smtp', 'All recipients refused: %s', e) # If the exception had an associated error code, use it, otherwise, # fake it with a non-triggering exception code errcode = getattr(e, 'smtp_code', -1) errmsg = getattr(e, 'smtp_error', 'ignore') for r in recips: refused[r] = (errcode, errmsg) failures.update(refused) ------------------------------------------------------ Mailman-Users mailing list Mailman-Users(a)python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/gamelover9001%40yahoo.… Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp --------------------------------- Yahoo! Music Unlimited - Access over 1 million songs. Try it free.

19 years, 2 months

Mailman-Users search results for query "ezmlm"