On 01/19/2015 08:30 AM, Gary Merrill wrote:

Second, this is a mailing list for a (loosely run) community band, this user is the president of the band and needs access to the mailing list. Implementing a Mailman list had as its primary goals addressing the problems involved in this user's maintaining his own private copy of his own list -- which was always fraught with error and often out of date. So having him be able to EASILY get the list is pretty important. It has taken over six months to get him to switch from his own private list to using the Mailman managed list. The rest of the band is ecstatic about the Mailman list, since they're now confident that it's always accurate and up to date, and that they won't miss any messages.

I suggest the easiest thing is to direct him to the list info page at a URL like http://example.com/mailman/listinfo/LISTNAME, enter his email address and list password in the Address: and Password: boxes near the bottom of the page and click the 'Visit Subscriber List' button.

Unfortunately, if his address or password is incorrect, all he sees is

Error LISTNAME roster authentication failed.

But again, the big GENERAL question here is whether Mailman provides any way of diagnosing such command failures.

If he is using the email "who" command, he should get an email response. Have him forward both his original email and the response email to you. Between the two, they contain everything Mailman knows about it. Mailman is designed to help the user find the mistake in this case, not to log everything for an admin.

You can search the web server and Mail server logs to find if he is mailing LISTNAME-request@... or POSTing to /mailman/roster/LISTNAME, but you won't be able to see the actual commands or data he's submitting.

You may ultimately find that actually sitting down with him is the easiest thing to do. I can give you patches to log everything if that's what you want, but walking him through it may be easier assuming he can remember for the next time.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

The suggestions you've offered have been tried (at this point multiple times). I was really asking whether there is a way to diagnose command failures in Mailman, and it's clear at this point that the answer is "No." This isn't entirely unexpected, but I just wanted to check in case there happened to be something that I'd missed.

A "diagnosis" of

Error LISTNAME roster authentication failed.

is like the diagnoses you used to see in old compilers that would helpfully inform you of "Syntax error" or "Invalid expression" (sometimes without even a line number). One wonders why anyone would ever write a utility with that sort of "error reporting", but it was quite fashionable for years. Mostly it came from developers writing things for themselves rather than for people who would use their products -- or for people whom the developers thought were like themselves, who mostly didn't make mistakes, and who were man enough not to need diagnostics when they did. Those were the days when men were men. Whatever. It is what it is.

Personal consultation and observation appears to be the only approach here that will be successful.

Gary H. Merrill Chatham Design Consultants +1 919.271.7259

-----Original Message----- From: Mailman-Users [mailto:mailman-users- bounces+ghmerrill=chathamdesign.com@python.org] On Behalf Of Mark Sapiro Sent: Monday, January 19, 2015 6:09 PM To: mailman-users@python.org Subject: Re: [Mailman-Users] Diagnosing command failures

On 01/19/2015 08:30 AM, Gary Merrill wrote:

...

Second, this is a mailing list for a (loosely run) community band, this user is the president of the band and needs access to the mailing

list.

...

Implementing a Mailman list had as its primary goals addressing the problems involved in this user's maintaining his own private copy of his own list -- which was always fraught with error and often out of date. So having him be able to EASILY get the list is pretty important. It has taken over six months to get him to switch from his own private list to using the Mailman managed list. The rest of the band is ecstatic about the Mailman list, since they're now confident that it's always accurate and up to date, and that they won't miss any messages.

I suggest the easiest thing is to direct him to the list info page at a URL like http://example.com/mailman/listinfo/LISTNAME, enter his email address and list password in the Address: and Password: boxes near the bottom of the page and click the 'Visit Subscriber List' button.

Unfortunately, if his address or password is incorrect, all he sees is

Error LISTNAME roster authentication failed.

...

But again, the big GENERAL question here is whether Mailman provides any way of diagnosing such command failures.

If he is using the email "who" command, he should get an email response. Have him forward both his original email and the response email to you. Between the two, they contain everything Mailman knows about it. Mailman is designed to help the user find the mistake in this case, not to log everything for an admin.

You can search the web server and Mail server logs to find if he is mailing LISTNAME-request@... or POSTing to /mailman/roster/LISTNAME, but you won't be able to see the actual commands or data he's submitting.

You may ultimately find that actually sitting down with him is the easiest thing to do. I can give you patches to log everything if that's what you want, but walking him through it may be easier assuming he can remember for the next time.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman- users/ghmerrill%40chathamdesign.com

Well, let's think about this ...

It's NOT true that "The user who sends an email command is provided with a response that tells him what's wrong." The example in question is where a valid user sends (from his registered address) the command 'who PASSWORD' but employs the wrong password or mistypes what he thinks is the correct password. In that case, the response is

      > You are not allowed to retrieve the list membership

This not only DOESN'T tell him what's wrong (Why am I not allowed to retrieve the list membership?), but is actually FALSE. As a registered user sending from the correct email address, he IS allowed to retrieve the list membership. But the message suggests that somehow he (as a registered user) doesn't have the correct permission to execute the command. That's just misleading, confusing, and wrong.

What would I want you to do? Well, how about a diagnostic of

      > Your request for the list membership has been denied because you

submitted the incorrect password for your account. > The password you submitted was: PUT_SUBMITTED_PASSWORD_HERE. > This is not your password for the PUT_LIST_NAME_HERE list.

This message is both informative and potentially helpful both to a user and to a remote administrator, as in the case at hand. Would it be a "security risk"?

Now let's look at the claim that "If the roster is not public, we can't give more information because if we tell you that it is the password that's invalid or the email address, this information can be used to fish for list membership and this is not public information." But this is specious reasoning. Why?

An astute user (which we presume would be the case with someone who's fishing for list membership) can discover whether the problem is with the password or the email address. (Actually the degree of astuteness here is not high.) The first thing he needs to do is determine whether he has a valid email address for a member. This is simple: he just fishes on the email address. He sends the command

      password

If he somehow manages to send this from the correct email account (or in some other more sophisticated way manages to send the command and catch the return message), he's rewarded by the list sending him that password. Now he can get the list membership. If he doesn't send this command from a member's registered email address, he gets the diagnostic message:

      > You are not a member of the XXXX list.

and he just keeps fishing. So not telling someone straightforwardly that they've submitted the wrong password doesn't add any degree of security against fishing. Anyone can quickly tell whether an email address is or is not one for a registered user, and then can quickly tell whether he does or does not have the correct password -- completely independent of whether you tell him that explicitly in a diagnostic message. Or have I missed something here? Actually, it seems to me that your response to a 'password' command from an unregistered email address is unnecessarily informative. It would be safer simply to respond with

      > Your request for your password has failed.  Please contact the

list administrator > immediately for additional information and to ensure the security of your account.

This at least doesn't say explicitly to the fisher that he's tried with a bad email address. But I think it's splitting hairs since the degree of security in any case here is fairly low.

So where are we now? It appears to be pretty clear that providing a more informative message in this case is both potentially quite helpful (to both the user and an administrator attempting to help him at a distance), and not a source of an increased security risk. So why not do it?

Finally, let's consider "The problem here is we don't anticipate a third party acting at a distance and unable to get reliable information from his user." Why not? This seems quite a reasonable scenario to anticipate. Users are notoriously unreliable in what they report and how they report it. And the user can provide information that's only as reliable and accurate as what he gets from the list in terms of diagnostics. I feel a pontificating lecture coming on about user interaction design, but I'll refrain :-).

At this point it's quite clear that the problem with this user is that he keeps using the wrong password. I've told him that, but I somehow have to (nicely) rub his nose in it. Without setting up an appointment with him and travelling about 50 miles round trip (which is going to be unlikely with him), there's not much I can do. But the simple change I've suggested in this one particular diagnostic response would allow me to help him remotely.

Gary H. Merrill Chatham Design Consultants +1 919.271.7259

-----Original Message----- From: Mark Sapiro [mailto:mark@msapiro.net] Sent: Tuesday, January 20, 2015 11:01 PM To: ghmerrill@chathamdesign.com; mailman-users@python.org Subject: Re: [Mailman-Users] Diagnosing command failures

On 01/20/2015 06:59 PM, Gary Merrill wrote:

...

I was really asking whether there is a way to diagnose command failures in Mailman, and it's clear at this point that the answer is "No." This isn't entirely unexpected, but I just wanted to check in case there happened to be something that I'd missed.

It depends. The user who sends an email command is provided with a response that tells him what's wrong.

We try to provide diagnostic information that is helpful to the user. The problem here is we don't anticipate a third party acting at a distance and unable to get reliable information from his user.

...

A "diagnosis" of

...

Error LISTNAME roster authentication failed.

What more would you like to see? Presumably the user knows what he provided for his email address and password and the response says one or both is invalid. If the roster is not public, we can't give more information because if we tell you that it is the password that's invalid or the email address, this information can be used to fish for list membership and this is not public information.

You could set Privacy options... -> Subscription rules -> private_roster (aka Who can view subscription list?) to Anyone and that would simplify your user's task, but you may not want a public roster.

...

Personal consultation and observation appears to be the only approach here that will be successful.

As I said,

...

...

I can give you patches to log everything if that's what you want

Let me know.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

If he can't get that to work, perhaps you could schedule a script to run it yourself and email him the results, say once a week.

Peter Shute

Sent from my iPad

On 22 Jan 2015, at 3:42 am, Mark Sapiro <mark@msapiro.net> wrote:

And, you could send him an email with a <mailto:LISTNAME-request@...?subject=who%20PASSWORD> link in it hand have him just click that and send.

I don't think this is the problem. So far as I know he has been using single-word passwords with only alphabetic characters in them. I'm really surprised that there would have been a bug involving initial or terminal whitespace, but I suppose these things happen. (Trying to imagine how that could have been left out of the tokenizer -- can't quite do it :-) ). We are using version 2.1.18-1 -- which appears to be the most recent stable release.

I do appreciate your efforts to help, but you should realize that some of them are based on assumptions that simply don't hold in this case -- although they pretty universally held when Mailman was originally written, and for some time afterward. The primary underlying assumption is that the list administrator will have access to at least most aspects of the OS and server environment. In this case, that isn't true.

We subscribe to web hosting from Webhostingpad.com. The level of our service does not give me a virtual machine, doesn't provide me with root access, doesn't even provide me with a shell, and does not provide me with direct access to any databases (I can create them and layout the tables, but that's about it). So I can't look at server logs and can't run shell scripts. I can't throw SQL at a db. I can't see any of the file system that's outside of my "home" directory tree. So I can't even see where any of the usual Mailman files are. I can only manage things through the Mailman web interface. Oddly, I can create cron jobs, but really have no way of testing/debugging them aside from creating the job and seeing what happens. Mailman lists are offered as one of the utilities under the Mail heading in Cpanel. It's simple. It works.

So suggestions about checking server logs, or the database, or running a shell script just aren't things I'm able to do. There's really no point in our having a higher degree of service since (whenever I turn this over to someone else), the level of knowledge of whomever ends up managing it after I do will be very low -- and may not include any *NIX experience at all. I'm trying to create an integrated web site and set of communication utilities that are virtually self-maintaining, other than having to modify web pages now and then, and maybe having to manage/configure mailing lists and users and such. For this reason, I am not interested in patches or custom code -- much as I love Python myself. In our entire band I would estimate that there are maybe three people who know what Python is, and I'm one of them. Any C or C# or C++ programmers? Maybe one or two others. Any really competent ones? Highly doubtful. Maybe one other.

Now you might argue that Mailman was never intended to be deployed in such a lame support environment. That is almost certainly true. But it's ALMOST good enough to stand on its own feet in this environment, and I think that a number of subscribers to Webhostingpad make use of it. It's been working very reliably and well with this one exception -- which is certainly the problem of this single user. Since Mailman doesn't log failures and doesn't return specific enough diagnostics to tell exactly what's going on, I'll have to deal with it a more direct and personal way. I think the best thing to do would be to get one of his buddies to sit down with him and show him how to do it and what his problem is.

Thanks for the suggestions.

Gary H. Merrill Chatham Design Consultants +1 919.271.7259

-----Original Message----- From: Mailman-Users [mailto:mailman-users- bounces+ghmerrill=chathamdesign.com@python.org] On Behalf Of Mark Sapiro Sent: Wednesday, January 21, 2015 9:04 PM To: mailman-users@python.org Subject: Re: [Mailman-Users] Diagnosing command failures

On 01/21/2015 08:42 AM, Mark Sapiro wrote:

...

And, you could send him an email with a <mailto:LISTNAME-request@...?subject=who%20PASSWORD> link in it hand have him just click that and send.

I assume you can find his password with something like

bin/dumpdb lists/LISTNAME/config.pck | grep <his email>

since your original request was for logs.

It occurs to me that possibly he is having so much difficulty because his password contains leading or trailing white space. I don't think this is possible with recent versions, but there have been bugs in this area in the past. What is your Mailman version? have you seen his password with something like the above?

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman- users/ghmerrill%40chathamdesign.com

Gary Merrill wrote:

Now you might argue that Mailman was never intended to be deployed in such a lame support environment. That is almost

It sounds like a fairly typical mailman support environment.

certainly true. But it's ALMOST good enough to stand on its own feet in this environment, and I think that a number of subscribers to Webhostingpad make use of it. It's been working very reliably and well with this one exception -- which is certainly the problem of this single user. Since Mailman doesn't log failures and doesn't return specific enough diagnostics to tell exactly what's going on, I'll have to deal with it a more direct and personal way. I think the best thing to do would be to get one of his buddies to sit down with him and show him how to do it and what his problem is.

A lot of people use the cPanel version and can't do anything complicated for support. Even though this particular case is about trying to help a difficult user to do something that's normally straightforward that's probably pointless anyway, I hope v3 at least gives cPanel users access to more diagnostics than v2 does.

Peter Shute

Peter Shute writes:

A lot of people use the cPanel version and can't do anything complicated for support. Even though this particular case is about trying to help a difficult user to do something that's normally straightforward that's probably pointless anyway, I hope v3 at least gives cPanel users access to more diagnostics than v2 does.

Don't hold your breath. The problem is hosting services (and to some extent cPanel itself), not Mailman. If they wanted to provide the kind of service you want, they could ask us for help and we'd give it to them. (Eg, it probably wouldn't be too hard to grep for specific vhosts in the logs, and so minimize the probability of cross-vhost personal information leakage. Of course, if they wanted to do that they probably already can. Evidently they don't care. :-( ) But if we added an option to (say) display the "smtp" log (or a hypothetical "auth" log for the current thread) the first thing they'd do is disable it in Defaults.py for the cPanel version since it would be too likely to leak personal data across virtual domains.

We do have some work (courtesy Andrew Stuart) ongoing to improve authentication for access to core databases in Mailman 3. I don't know exactly what that work is at the momemnt, but it might be possible to leverage it so that subscribers and/or list owners could use Mozilla Persona or OAuth2 to access the information they're authorized for. (Postorius already requires, or soon will require, Persona for authorization IIRC.) At least then you can pass the buck for authn/authz issues to Google or Mozilla. ;-)

If you have *specific* diagnostics that don't violate the hosting services' senses of propriety (and I suppose they normally won't be problematic), please, *please*, PLEASE *ask* for them by name (a brief description is fine, too, of course). "More diagnostics" is not a requirement we can use to specify and design new features.

Preferably a bug report to Launchpad, tagged "mailman3", but a post to the list or a request on IRC would probably be picked up, too.

Again, thanks to all of you for your responses. There are a number of thorny issues in attempting to "fix" (or at least address) what is "wrong" here. The original conceptual model of something like Mailman isn't quite right for the situation being faced today in the types of capabilities and support offered to the "general public". This doesn't mean that that model was wrong, but that's it's not a thoroughly good fit for a certain service/support/marketing model employed by others. Making things "fit" better is a daunting task because of the numbers and types of players involved, and their varied goals and constraints. Hey, before I retired, I spent about 13 years developing knowledge representations and software (server and user) for multiple disparate medical coding schemes for the purposes of drug discovery and adverse event detection. I know the kinds of problems you guys are facing -- although you should be somewhat grateful that you aren't dealing with

companies, gigantic healthcare organizations, the CDC, and multiple pharma companies. :-) "Don't hold your breath" is a

On 1/22/2015 10:10 AM, Gary Merrill wrote: the FDA, insurance perfectly apt comment.

The hosting services companies may want to provide certain services

(or at least their developers and support staffs may). They (meaning the their developers and support organization) may in fact care, but they may be severely constrained in what they can do within the economic model of the company and how effectively they can fight their marketing organization. The issues here often aren't technical, but economic and political (in the sense of company politics). Sometimes the good guys win, and a lot of times they don't. And to some degree you get what you pay for (if you're lucky and deal with a reputable provider). I'm quite happy with the service and support that Webhostingpad has provided me (I have two sites hosted by them).

Prior to running the Mailman list for this community band, I had --

um on a regular basis "couldn't remember" to do so, etc. Basically, it was something that a contingent of the organization wasn't familiar with and just didn't want to learn about -- whatever the advantages were. It gave me a lot of flexibility and control because virtually everything concerning its administration was in "my own space". As in the case with Mailman, I avoided any patches, custom code, or addition of anything other than the base modules and some images. I had to abandon it because as a communication mechanism, it just wasn't working for this group of people. No amount of education or support would solve the problem.

I then recalled the experience I'd had with the listservs used by the National Library of Medicine and thought: That should do the trick for

for a year -- been running a phpBB bulletin board. Of course this is quite different from Mailman, is more feature-rich, employs a completely different conceptual model and architecture, and required that I install and configure it myself. That gave me a lot more control over it than I have over my Mailman lists that are available through Cpanel. It was a noble experiment that failed. While the majority of the band really liked the idea and functionality of forums, there was a contingent that was highly resistant to using that approach. (These were not always, or even generally, the elder members, but somewhat oddly tend to be in the 45-55-ish age group.) They didn't like having to go to the site ... they didn't like having to employ a password to access it ... but they wanted to be sure that everything was secure and not visible to outsiders ... the primary people who in fact needed to make postings to the for this group. Their model of online communication is email. It's simple. Of course, 80% of my organization can't distinguish between a privately held and maintained mailing list of contacts and a mailing list managed by something like Mailman -- but it's working out nicely and they're learning the few important fundamentals over time. By far, most of them have never changed their default password (everyone originally was batch-subscribed) or even used their password or felt the need to. Many of them have probably forgotten that they even have a password.

...

From my point of view, what would be ideal in managing a Mailman list

would be to have all the relevant data for the list in "my space", and accessible to me so that on the rare occasions when things don't work I could at least look for what is causing the problem. Some configurable level of logging (again with the information kept in "my space") would be very helpful so that this could be turned on and off as necessary. Perhaps administrator-configurable/editable error messages would be nice as well -- and ones not requiring modifying code or central files governing the entire Mailman instance.

That is, some enhanced degree of "local list control" would be very

helpful. Currently this seems to be supported only through the ability to edit "public" HTML and text files (of which there are few) -- so someone has recognized the utility of local list control, but it's just never been taken very far. This is all to say that I think much would be gained by moving the perspective of "the administrator" from a global perspective to a more list-local perspective in many instances. Perhaps the upcoming release will include such things; perhaps not. I don't know your architecture or your constraints, and so I'm not going to start pitching such ideas at you in the form of change requests.

Mailman is currently quite nicely satisfying the needs I have, and I

look forward to the next release.

---

Gary H. Merrill Chatham Design Consultants +1 919.271.7259

One quick comment on Mailman support - When I was managing a Mailman installation on Ubuntu, I was told that I had to install a package. I looked at the Ubuntu package and compared the source therein with the SourceForge source. There were many changes, and I could not determine what they all were. My boss told me that we could install the Ubuntu package and then get a support contract with Debian/Ubuntu. I told him that I was more comfortable with the SourceForge source and the support I got from this list. He allowed me to build a Mailman package from the SourceForge source, and that is what I did. I never installed the Debian/Ubuntu Mailman package, so I never had to talk to the Debian/Ubuntu Mailman support staff.

--Barry Finkel

Gary Merrill writes:

Personal consultation and observation appears to be the only approach here that will be successful.

If the security implications are acceptable (which I think they are), you can also provide the user with a precomposed URL that they can put in their bookmarks list, as Mark mentioned.

We don't provide that kind of thing ourselves because it is a security risk, and also in some contexts a legal risk (some list owners are subject to legal requirement to get confirmation for subscription, for example).

Aside: in this case it's not necessary AFAIK, but in cases where a CSRF token or similar dynamically generated security token might be involved, it should be possible to use Java script to acquire and return the token.