Michael B. Trausch writes:
> My main, central, and driving point is a desire to create a system that
> is both idiot friendly and expert friendly, and my desire is to have all
> known common (end-user) and uncommon (technical user) use cases be
> handled in a manner that is consistent and portable.
That's what we want, too, Brother Michael. The difference between you
and us is that we believe in the RFC process, we believe in
decentralization, and we believe that if we design a better way to do
it in that context, software developers will adopt it and users will
get the benefits merely by upgrading their favorite software (or
changing, if another implementation becomes more attractive in the
process).
Summary: Spammers now have so many ways of "harvesting" addresses from so
many systems, and so many ways of exchanging those with each other, that
any email address which is actually used WILL eventually be harvested.
(Where what "eventually" means varies widely, of course, but can be
expected to steadily decrease.) Pretending that address obfuscation
in mailing list [or newsgroup] archives will have any meaningful effect
on this process gives users a false sense of security and has zero
anti-spam value.
Summary of summary: It's pointless.
Explanation: Spammers maintain extensive databases of email addresses.
Some of those databases are merely lists of addresses; others are
more sophisticated and include data such as "harvested-date",
"havesting-method", "last-seen date", "last-seen-context",
"last-known-valid date" and more. Some of these databases are private;
others are available for sale/lease. Some are maintained by spammers
themselves, others by spammer support services don't directly engage
in spamming.
The harvesting engines used to acquire email addresses are myriad, as
are the methods by which spammers acquire the raw data to use as input
to them. *Some* of those methods, and there are many more, include:
- subscribing to mailing lists
- acquiring Usenet news (NNTP) feeds
- querying mail servers
- acquiring corporate email directories
- insecure LDAP servers
- insecure AD servers
- use of backscatter/outscatter
- use of auto-responders
- use of mailing list mechanisms
- use of abusive "callback" mechanisms
- dictionary attacks
- construction of plausible addresses (e.g. "firstname.lastname")
- purchase of addresses in bulk on the open market.
- purchase of addresses from vendors, web sites, etc.
- purchase of addresses from registrars, ISPs, web hosts, etc.
- domain registration (some registrars ARE spammers) [1]
- misplaced/lost/sold media (hard disk, tape, CD, DVD,
USB stick, etc.)
and perhaps most significantly:
- harvesting of the mail, address books and any other files
present on any of the hundreds of millions of compromised
Windows systems [2]
Consider for example: the first time a newly-created address is used
by someone (who is sending a message to it), it's now present on their
system: in their saved outbound mail, or perhaps in their address book
(if they have one), or in some cache. Any sensible malware resident on
their system will of course pick it up and eventually hand it over to a
harvesting agent. (Competent malware will harvest it in real time *and*
associate it with the sender's address.)
And if that particular system happens to be clean? Doesn't help much,
because the more times that address is used, the more systems it's
present on. And the more systems it's present on, the greater the
probability that one of them is already compromised or will be soon.
Thus even if we eliminate the originating end-user system as a possible
source, we still have to consider the outbound mail server used by that
end-user system, which is also a candidate for compromise. And then
the inbound mail server used by the recipient, and then the recipient
end-user system. And if there's some filtering appliance or intermediate
system in place at either end, then it's there too. If the message
is forwarded to a third party, then another set of systems is in play.
If mail server logs are rolled up and moved to some central location,
then it's there too. If backups are made, then it's present there,
and subsequently may be present on any system where the backups are
read/restored. And finally, if the destination of a mail message isn't
an individual user, but an entire mailing list, then we must multiply
the number of possible harvesting points by at least the number of people
on the mailing list plus a factor for mail servers/gateways/filters/etc.
(modulo overlaps). This in turns means that messages to sent to lists
of any appreciable size (say, 1000 members) will turn up on considerably
more than 1000 systems -- and the chances that all 1000-plus are secure
are microscopic.
Please note that the previous paragraph's recitation only covered the last
vector I enumerated in the [indented] list above: compromised systems.
That laundry list of methods also affords many other opportunities for
addresses to find their way into spammers' hands. As just one pointed
example out of a great many more that could be cited: how do you know that
the address user(a)example.com which has just subscribed to the list you
run is a real person and not just the front-end for an address-harvester
that will pick up every address used to send traffic to the list?
And so on. There are far too many others to enumerate, all of which
have discussed at great length in anti-spam forums for many years, and are
depressingly familiar to experienced practitioners working in the field.
The bottom line is that any email address which is actually used [3],
*especially* any email address used to send traffic to a mailing list,
is going to be harvested. It's only a matter of when, not if, and "when"
is getting sooner all the time.
Incidentally, everyone (including me) can produce anecdotal tales of
addresses that have remained surprisingly under-targeted by spammers
over long periods of time. But this is clearly not the way to bet: it
is in spammers' interests to ferret out as many addresses as possible
and to use them as soon and as often as possible. Note, however,
that some addresses are *deliberately* un-/under-targeted, so lack of
substantial spam traffic to a given address is NOT an indicator that the
address hasn't been harvested. That's because along with target lists,
spammers maintain "suppression" lists, which they use to avoid hitting
the addresses of people they think are likely to cause issues for them. [4]
And obviously, people with postmaster or mailing list roles would be
good candidates for membership on those lists. I know that if I were in
their shoes, I'd add everyone who's ever sent a message to the mailman-*
mailing lists to mine: a quick check indicates that it's on the order
of only 10K addresses. Skipping those would be inconsequential when
sending spam to a few hundred million addresses, and I trust it's
obvious why spammers would benefit from doing so.
With all this in mind, it's clearly pointless to pretend that address
obfuscation in archives provides any protection at all. [5] It would be
better to remove the code entirely than to continue to maintain the
facade that it actually has any anti-spam value. Everyone should simply
presume that all email addresses are in the hands of spammers and prepare
defenses accordingly -- because even if that's not quite true yet, it will
be soon enough.
Notes:
[1] I deliberately didn't mention mass WHOIS queries. While some efforts
in this direction were made by spammers years ago, they've found it far
more efficient and cost-effective to simply buy WHOIS data in bulk.
There's always someone who wants to sell, and a CD/DVD or USB stick
will suffice. This is why attempts by registrars to rate-limit queries
or restrict access are not only foolish, but disengenuous: spammers
already *have* the data, and can acquire updates at will, and they
are clearly doing so via processes that lead back to registrars themselves.
[2] The exact number of such systems is not only unknown, but unknowable,
since any compromised system which (a) doesn't make its presence
known (b) to a suitable detector will remain undetected indefinitely.
However, two things are clear: (1) any estimate under 100 million should
be laughed out of the room, and (2) there is no reason to suspect that
the number is decreasing, and there are numerous reasons to suspect that
it's increasing. Note, incidentally, that some detectors have reported
observing 200,000 new such systems in a single day; and further note that
it's now quite routine for individual botnets with several million
*known* members to turn up.
[3] Addresses which aren't used may remain out of spammer view for
considerable time, depending on the care with which they're selected
and maintained. However, this obviously excludes addresses used for
participation in mailing lists.
[4] For the purpose of this discussion, I'm just talking about
suppression lists which enumerate individual email addresses. It's
well-known that spammers also maintain suppression lists of MX's, domains,
network allocations, ASNs, etc., in an attempt to avoid hitting
spamtraps and/or hitting the mailboxes of those who might be in a position
to file complaints or take action against them.
[5] The only people left who are impeded in the slightest by obfuscation
code are NON-spammers: that is, people who are trying to contact someone
who has previously sent a message to some mailing list.
---Rsk
I'm happy to announce the release of Mailman 3.0 alpha 4.
Things are getting quite real now. It's pretty easy to hook Mailman
up to Postfix for both incoming and outgoing mail[1] and you can
create mailing lists and populate them with addresses. Currently, you
must do this through the command line though.
Please download the latest tarball or grab the bzr branch, and put it
through some paces. Now is a good time to comment on the design and
feature set for Mailman 3 as we are still in alpha. I'm still planning
on releasing a first beta by the end of the year -- your testing,
feedback, input, and contributions can help with that!
You can download the tarball either from the Cheeseshop, or Launchpad:
http://pypi.python.org/pypi/mailman/3.0.0a4https://launchpad.net/mailman
Please note: Mailman 3.0a4 is not production ready yet.
Enjoy,
-Barry
I just sent a message to my freshly installed MM 3 alpha 4 (on Ubuntu 9.04)
and got the following warning:
Nov 29 23:53:51 mailman postfix/lmtp[8620]: D409D32E12: to=<test(a)mailman.state-of-mind.de>, relay=127.0.0.1[127.0.0.1]:8025, delay=1.8, delays=0.07/0.01/0/1.7, dsn=2.0.0, status=sent (250 Ok)
Nov 29 23:53:51 mailman postfix/qmgr[8613]: D409D32E12: removed
/root/mailman/src/mailman/rules/administrivia.py:84: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if line == '':
The delivery worked though.
p@rick
--
state of mind
Digitale Kommunikation
http://www.state-of-mind.de
Franziskanerstraße 15 Telefon +49 89 3090 4664
81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
Hi everybody,
I'm getting very near ready to release another alpha of Mailman 3, and on the
prompting of a private message from Robert Niederreiter I took some time to
fire up a VM and actually try Postfix+LMTP delivery in an experimental
production system. I'd like to get some feedback from the Postfix experts in
the crowd so that we can update the wiki page here:
http://wiki.list.org/display/DEV/LMTP+process
FTR: the VM is running Ubuntu Karmic Koala 9.10 server with Postfix 2.6.5 and
Mailman 3.0 from the lp:mailman bzr trunk.
To start with, I mostly followed the instructions in the wiki. I tried using
the standard lmtp transport in master.cf, creating /etc/postfix/mailman_lists
as described in the wiki (not the dedicated list server example, but the
earlier one on the page). I actually used:
# Key # Value
test(a)xxx.example.com lmtp:xxx.example.com
where 'xxx.example.com' is replaced by my VM's host name.
The first gotcha is that unless you start Mailman as root, you can't bind its
LMTP server to port 24, which is Postfix's default. By default Mailman's LMTP
server listens on localhost:8024, so you either need to append ':8024' on the
Value above, or set
lmtp_tcp_port = 8024
in Postfix's main.cf. I ended up doing the latter, but both work.
I set up transport_maps and local_recipient_maps just as outlined in the wiki
page, fired everything up, and then sent a message to the VM's Postfix while
tailing the logs. I kept seeing "Connection refused" messages from Postfix
and it never hit Mailman's LMTP server.
This was highly confusing because I could see in the Postfix logs that it was
finding the right IP address for its own hostname and it was trying the right
port number. Mailman's lmtp.log file claimed it was listening on the right IP
and port, and I could even telnet to it just fine. So what was Postfix doing?
It seems that Karmic is playing tricks with /etc/hosts. I've got DNS set up
to correctly resolve the VM's hostname, but there was actually an entry in
/etc/hosts that set xxx.example.com to 127.0.1.1. I'm not entirely sure which
process looks at what, when, but clearly this inconsistency was a key part of
the problem. The other thing that surprised me was that Postfix was also
consulting /var/spool/postfix/etc/hosts, and I had to change both /etc/hosts
and that file as well, so that the IP addresses jived with DNS. This is
unsatisfactory, but it seemed critical to getting things working.
The last piece of the puzzle was to not use Postfix's standard lmtp server in
master.cf, but to define a new one like so:
mailman3 unix - - - - - lmtp -o disable_dns_lookups=yes
and then change the mailman_lists transport map to
# Key # Value
test(a)xxx.example.com mailman3:xxx.example.com
After a restart, everything suddenly worked exactly as expected.
Robert was having a different problem, but hopefully he will follow up here
with his experience and let us know if any of the above helps. If any Postfix
experts have words of wisdom to make this better, please let me know.
I probably need to work on better dropping of privileges for qrunners so that
you can 'bin/mailman start' as root, and once the LMTP runner binds to port
24, it can drop privs to 'mailman'. I'll put that on the list for something
to do after the next alpha.
I'd also like to try to resurrect William Mead's LMTP branch. Sadly, it won't
merge cleanly into trunk any more (not the least of which because it's in the
wrong bzr format). If anybody would like to contribute that before I get to
it, I'd be grateful.
The good news is that I think Mailman 3 is getting more real every day. My
plan over the next few weeks is:
* release alpha 4
* get i18n translations working
* complete the split of the pipermail project
* hopefully get Patrick and friends going on the web u/i
* announce something <wink>
* create a live test list that anybody can subscribe to
If you've been waiting to play with Mailman 3, wait no longer!
-Barry
MM developers,
I'd like to propose a change in MM3s default SMTP client port from port 25
(transport) to port 587 (submission).
Why? From my point of view mailman rather is a mail component that introduces
messages into a mail system than one that sits between MTAs and assists in
transporting messages that pass by.
RFC 4409 <http://www.rfc-editor.org/rfc/rfc4409.txt> explicitly defines a
submission port (587) for mail systems whose purpose is to accept message from
MUAs:
However, SMTP is now also widely used as a message *submission*
protocol, that is, a means for Message User Agents (MUAs) to
introduce new messages into the MTA routing network. The process
that accepts message submissions from MUAs is termed a Message
Submission Agent (MSA).
Apart from doing 'the right thing' what would be the benefit?
The RFC gives some ideas in a later section:
(...) Even when submitted messages are complete, local site policy may
dictate that the message text be examined or modified in some way, e.g., to
conceal local name or address spaces. Such completions or modifications
have been shown to cause harm when performed by downstream MTAs -- that is,
MTAs after the first-hop submission MTA -- and are in general considered to
be outside the province of standardized MTA functionality.
>From my daily work with mailman the following "modified in some way"-tasks
come to my mind immediately:
- apply client and content policy that differs from the port 25 anti-spam
policy
- add DKIM signatures because it is clear mailman messages are ORIGINATING
from our network
What would we have to do, to make port 587 the default port? In section 4 the
RFC says, a MSA MUST do all of the following:
1. General Submission Rejection Code
2. Ensure All Domains Are Fully-Qualified
3. Require Authentication
To cut it short: 1. and 2. are trivial (at least in Postfix and I don't know
the others MTAs well enough to tell for them too). 3. requires to add SMTP AUTH
functionality to Mailman's SMTP client.
How should we implement SMTP AUTH in the MM SMTP client?
I propose for a start plaintext (PLAIN, LOGIN) and shared-secret mechanisms
(CRAM-MD5, DIGEST-MD5) should be added to the SMTP client. Those are the ones
used most widely in every day SMTP AUTH.
Later implementations could add GSSAPI and EXTERNAL. If plaintext mechanisms
are added we should also consider to add STARTTLS functionality to MM's SMTP
client to shield credentials while they are sent in a plaintext authentication
session.
p@rick
--
state of mind
Digitale Kommunikation
http://www.state-of-mind.de
Franziskanerstraße 15 Telefon +49 89 3090 4664
81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
Hi,
I´m new with Maiman.
I need make mail of subscription with multiple users,
Is possible make a mail with name-request(a)sudominio.com with several lines
of subscribe [contraseña] [digest|nodigest] [address=] command
if is yes, I need a example,
if is not possible, as can be done?
thanks,
Sorry for my inglish
--
View this message in context: http://old.nabble.com/Mail-of-subscribe-to-mailing-list-with-multiple-users…
Sent from the Mailman - Dev mailing list archive at Nabble.com.
As you know, Mailman 2.1 has long been in maintenance-only mode.
Mailman 2.2 was where we were going to add new features and update the
user interface, without changing the basic model. Mailman 3 was where
we were going to fix the model and modernize the architecture to allow
for better embedded use. Mark has been doing an incredible job fixing
Mailman 2.1, and forward porting these fixes to Mailman 2.2. I have
been working on Mailman 3 and have released several alphas.
The current state of affairs is not ideal though. Neither 2.2 nor 3.0
has been released, there is confusion in the community as to which
version to develop patches for, and frustration on our part that we
have divided efforts and not as much community participation as we'd
like.
Mark and I have decided therefore to combine our efforts under Mailman
3, and we invite you to join us. Working together, I feel confident
that we can have a solid release of Mailman 3 very soon, hopefully by
the end of the year. Patrick Koetter and his group have expressed
interest and resources in helping jump start the new Mailman user
interface, which will be built on top of Mailman 3's REST interface.
What do /you/ want to work on? :)
Here's the plan: Mark is going to put a 2.1.13 bug fix release out
soon and will continue to fix only the most important bugs on the 2.1
branch. He'll forward port those fixes to the 2.2 branch for the few
people who are running it from source, but there will never be a
Mailman 2.2 release. For all practical purposes, Mailman 2.2 is
dead. Mark will be joining me to focus all new development work on
Mailman 3.0.
I hope this brings clarity to where we're going, and I hope that the
renewed and concentrated efforts will encourage you to pull down the
Mailman 3.0 code or alphas and begin testing and developing for it.
Enjoy,
-Barry
Hi,
I've terminated to configure my server but when I create a list, the welcome
message stops on queue.
Analysing /var/log/mailman/smtp-failure I get:
Oct 29 09:04:29 2009 (18551) delivery to test(a)xxx.xxx.xxx failed with code
-1: (101, 'Network is unreachable')
Could anyone help?
Thanks
Wallace Gerheim
When the obscure email address feature in the admin settings is
enabled, e-mail addresses are transformed into the format 'xxx AT xxx'
in the archives. This may have worked 5 years ago to prevent spam
harvesting. But it does not work anymore.
I've patched (well... butchered) my own install by modifying
HyperArch.py to simply output 'EMAIL HIDDEN' instead of the obscured e-
mail address, by replacing the regexes to a constant string at line 284.
I think a 'proper' feature like this would be a very welcome addition.
Right now, it would be trivial to write a script to harvest tens of
thousands of e-mail addresses from Mailman users...
--
Joost Schuttelaar
The Hague, NL