Maybe I ought to explain what I'm up against. At work I'm running Lyris. I have hundreds of lists and many, many members. I also have Mailman running a handful of small-ish lists (and at home I run a server with Mailman for a personal interest discussion list of nearly 1,000 members). My organization (and in fact, my entire industry of international development) is embracing open source with a fervor and one of the things on the to-do list is to move all services that run on commercial software to open source software. NT, IIS, ColdFusion and Lyris, are, for all intents and purposes, history. I now have online communities that have been very much accustomed to Lyris features, performance, functionality, etc. Not to mention the web interfaces that marry Lyris and other online resources (profiles, documents, calendar, etc.). I have the unenviable challenge of moving these people to a Mailman environment without making them feel like it is a step backward. I would really like to believe that this is possible. I'm not ready to give up on this quite yet. I have looked at other open source MLM's, and for one reason or another, the others aren't at all usable. Mailman is the closest thing I see to a solution.
Anyway, that's my sob story.
I'll continue to look forward for solutions. Flame away if it'll make you feel like a better person, but I'm not giving up that easily.
- Kevin
On Fri, 2004-01-30 at 15:15, Kevin McCann wrote:
I'll continue to look forward for solutions. Flame away if it'll make you feel like a better person, but I'm not giving up that easily.
I for one, hope you don't give up Kevin!
Mailman's come a long way since its humble beginnings, and I enjoy seeing where people are pushing it as software, and as a project.
it-only-looks-like-a-long-way-down-ly y'rs, -Barry
On Jan 30, 2004, at 12:15 PM, Kevin McCann wrote:
I have the unenviable challenge of moving these people to a Mailman environment without making them feel like it is a step backward.
A laudible goal. And sometimes, the only way to make it possible is through some front end investment. Open Source isn't free, it simply doesn't have a price tag. the cost is in sweat equity and hours invested by interested parties.
So to leverage open source, you have three choices:
find packages and projects that are sympatico with your needs and interests, or at least come close enough to take advantage of them.
start typing -- dig in and do some sweating to take a package in the directions you need it taken.
influence and support others to do the sweating to your needs: in other words, patronage.
I'll continue to look forward for solutions.
there are always solutions. Not all of them are already packaged up and waiting to be found, or easily created through talk.
I'll continue to look forward for solutions.
there are always solutions. Not all of them are already packaged up and waiting to be found, or easily created through talk.
I understand this, Chuq. For me, the most pressing thing is the SQL activity. And as I have mentioned, we're ready to contribute resources to this end. Either I will get involved, if I can get up to speed technically, or we'll hire someone to get involved with Barry. I don't want to just talk about this stuff. I want to see things happen. I don't know precisely how much funding I can get for this, but I'll be doing what I can. But some discussion *is* necessary. I have been harping about SQL in Mailman for a year now but it is only recently that the mere thought of it being worked into MM in one way or another has being entertained. Talking isn't entirely useless. In fact it's necessary in order to influence change. Just so long as it doesn't stop at talk.
- Kevin
On Jan 30, 2004, at 9:46 PM, Kevin McCann wrote:
there are always solutions. Not all of them are already packaged up and waiting to be found, or easily created through talk.
I understand this, Chuq. For me, the most pressing thing is the SQL activity. And as I have mentioned, we're ready to contribute resources to this end.
Fair 'nuff. I thought you did, but it's stuff that has to be said once in a while, anyway.
I have been harping about SQL in Mailman for a year now but it is only recently that the mere thought of it being worked into MM in one way or another has being entertained. Talking isn't entirely useless. In fact it's necessary in order to influence change. Just so long as it doesn't stop at talk.
I've got a bit of MySQL background. My time is still limited, so I won't commit to coding I can't depend on myself to finish -- but if you want someone to help out on DB design and stuff, I'm in. I'll do what I can and maybe help avoid some of the potholes. I'd like to see this, too.
Chuq Von Rospach wrote:
I've got a bit of MySQL background. My time is still limited, so I won't commit to coding I can't depend on myself to finish -- but if you want someone to help out on DB design and stuff, I'm in. I'll do what I can and maybe help avoid some of the potholes. I'd like to see this, too.
Perfect.Any help would be appreciated. I'm packing it in for tonight (it has been a long day), but I'll be in sometime during the weekend. Maybe Barry has some ideas as to how things ought to move along wrt his mm3 work in progress. I'm wondering if some inroads can be made even before the sprint session in March. I can show you and Barry the DB stuff I had in mind and maybe we can jumpstart from there.
- Kevin
On Sat, 31 Jan 2004, Kevin McCann wrote:
I'm wondering if some inroads can be made even before the sprint session in March. I can show you and Barry the DB stuff I had in mind and maybe we can jumpstart from there.
I likewise had an abortive attempt to make the api changes necessary to fold in a more scalable sql backend, and would offer that as another example if we want to start these discussions before March. I'm dubious, though, that we'll get as much progress for the time investment doing this through email instead of in person...
Dale Newfield <Dale@Newfield.org>
"They that can give up essential liberty to obtain a little safety deserve neither liberty nor safety." - Benjamin Franklin, on the Statue of Liberty
On Sat, 31 Jan 2004, Dale Newfield wrote:
if we want to start these discussions before March.
Oh--I forgot to mention--I'll be joining Barry and whoever in March for Sunday and Monday of the sprint, and I've got a bunch of SQL experience... ...wasn't somebody asking if anybody fit that description?
-Dale
I feel bad about jumping in to this discussion, and not necessaraly being prepared to offer any real help, but anyway.
LDAP may be a better data[base|sink] for configuration data then SQL. It may be harder to work with (only because less people have experience with it), and it may be less usefull to a lot of sites (everyone has MySQL installed, not so for LDAP), but preference storage is a common use for LDAP. For sites who already have LDAP deployed, you can leverage the existing authentication data. LDAP stores "objects", SQL "rows"...
Sory if I sound like a marketdroid, but I'd feel bad if LDAP wasent at least considered.
Quoting Chuq Von Rospach <chuqui@plaidworks.com>:
On Jan 30, 2004, at 9:46 PM, Kevin McCann wrote:
I have been harping about SQL in Mailman for a year now but it is only recently that the mere thought of it being worked into MM in one way or another has being entertained. Talking isn't entirely useless. In fact it's necessary in order to influence change. Just so long as it doesn't stop at talk.
I've got a bit of MySQL background. My time is still limited, so I won't commit to coding I can't depend on myself to finish -- but if you want someone to help out on DB design and stuff, I'm in. I'll do what I can and maybe help avoid some of the potholes. I'd like to see this, too.
On Jan 31, 2004, at 10:53 AM, Jeff Warnica wrote:
I feel bad about jumping in to this discussion, and not necessaraly being prepared to offer any real help, but anyway.
LDAP may be a better data[base|sink] for configuration data then SQL.
actually, Jeff has a point. I was muttering to myself about this last night.
Mailman <-> LDAP as an interface means that anything that can generate an LDAP interface can talk to it. so perhaps the best thing to do is come up with an LDAP interface, define how the LDAP data should look, and then create a set of MySQL schemas that'll support that. I know barry's wanted to avoid requiring too many "things" to be installed to use Mailman, but when someone chooses to move to MySQL, I don't think it's unfair to assume they have or can install LDAP also.
And that would allow someone to use some other LDAP backing store more easily, since we're hiding the interface in a fairly standard setup. And with MySQL as the proof of concept, having somenoe add support for Postgres SQL, Oracle, DB files or something else would be a lot easier.
That'd also make life really rather nice for larger operations, because it's not far from that to splitting off the web piece from the delivery piece from the data store piece, allowing it to exist in organizations where services are cloistered from each other.
I like the idea. It's not much more work (if it's more work at all), and much more flexible, and creates an interface others can connect to as well, not a single vendor solution. I'll buy in...
On January 31, 2004 11:10 am, Chuq Von Rospach wrote:
Mailman <-> LDAP as an interface means that anything that can generate an LDAP interface can talk to it. so perhaps the best thing to do is come up with an LDAP interface, define how the LDAP data should look, and then create a set of MySQL schemas that'll support that. I know barry's wanted to avoid requiring too many "things" to be installed to use Mailman, but when someone chooses to move to MySQL, I don't think it's unfair to assume they have or can install LDAP also.
Isn't LDAP a bit of a security hassle? I would think it is pretty common to
have Mailman running on a machine along side MySQL, Apache and and MTA of
some sort but wouldn't throwing in LDAP be more like requiring people install
a CVS daemon to use Mailman? I'm no LDAP guru but from what I have looked at
previously it certainly seemed that way.
Cheers
-- ---> (culture) http://industrial.org : (label) http://deterrent.net ---> (community) http://ampfea.org : (hire me) http://codegrunt.com ---> (send EEEI news to) infosuck@industrial.org ---> Whomever dies with the most URLs wins!!!!!!!!!!!!!
Isn't LDAP a bit of a security hassle? I would think it is pretty common to have Mailman running on a machine along side MySQL, Apache and and MTA of some sort but wouldn't throwing in LDAP be more like requiring people install a CVS daemon to use Mailman? I'm no LDAP guru but from what I have looked at previously it certainly seemed that way.
it's secure enough that most major IS/IT organizations seem to be standardizing it for distributing information like this. We now have, in fact, an internal list server that does exactly this now, but I wasn't directly involved in that project (it was a rewrite of a system I wrote years ago by someone else). And in MacOS X 10.3, Apple started moving it's stuff from NetInfo to OpenLDAP.
I think if it's secure enough for companies to implement, we don't have many worries if we take care to use it properly.
I suppose it can be, but it is a question of where you implement your security. If mailman is to use SQL to store preferences then it is up to mm to deal with what records a user can update. If the mm interface to LDAP goes through one master LDAP account, then it is still mm's job... But if mm binds to LDAP as the mm user, then security is the responsibility of the LDAP server. With OpenLDAP, and NDS permissions can be extreemly fine grained, down to the attribute level. Ive not so much as seen ADS running anywhere, but I can only assume that it does too.
How secure an admin might want to make it is likely to be related to what else, if anything, their LDAP directory is being used for. A hypothetical site with 10,000 users in NDS, and 100,000 other things (printers, queues....), which they have been using for a decade, may be very restrictive. Another site installing MM+LDAP for fun as much as anything else, might just give the MM user unlimited rights.
Kinda like the Bugzilla install docs something to the effect of ".... MySQL's security is an evil beast, and some people actualy use it. If you do, just make sure that 'bugz' has the right rights...."
Quoting moron <moron@industrial.org>:
On January 31, 2004 11:10 am, Chuq Von Rospach wrote:
Mailman <-> LDAP as an interface means that anything that can generate an LDAP interface can talk to it. so perhaps the best thing to do is come up with an LDAP interface, define how the LDAP data should look, and then create a set of MySQL schemas that'll support that. I know barry's wanted to avoid requiring too many "things" to be installed to use Mailman, but when someone chooses to move to MySQL, I don't think it's unfair to assume they have or can install LDAP also.
Isn't LDAP a bit of a security hassle? I would think it is pretty common to have Mailman running on a machine along side MySQL, Apache and and MTA of some sort but wouldn't throwing in LDAP be more like requiring people install a CVS daemon to use Mailman? I'm no LDAP guru but from what I have looked at previously it certainly seemed that way.
On Sat, 2004-01-31 at 15:56, Jeff Warnica wrote:
I suppose it can be, but it is a question of where you implement your security. If mailman is to use SQL to store preferences then it is up to mm to deal with what records a user can update. If the mm interface to LDAP goes through one master LDAP account, then it is still mm's job... But if mm binds to LDAP as the mm user, then security is the responsibility of the LDAP server. With OpenLDAP, and NDS permissions can be extreemly fine grained, down to the attribute level. Ive not so much as seen ADS running anywhere, but I can only assume that it does too.
How secure an admin might want to make it is likely to be related to what else, if anything, their LDAP directory is being used for. A hypothetical site with 10,000 users in NDS, and 100,000 other things (printers, queues....), which they have been using for a decade, may be very restrictive. Another site installing MM+LDAP for fun as much as anything else, might just give the MM user unlimited rights.
It's things like this that give me the willies and keeps me up at night. It's already more difficult than I'd like for the average joe to install Mailman and integrate it with all the other moving parts. By using A Real Database, we have to accept that it will be even more difficult because there isn't any such db that I'm aware of that is fully transactional but requires no administration. Say one thing about MM2's crufty pickle storage, but it's brain dead easy and requires no administrative overhead. Of course it doesn't scale, which is why it's acceptable to add some overhead, but forcing most users to deal with stuff like the above is more than we should ask of them.
-Barry
On Sat, 2004-01-31 at 13:53, Jeff Warnica wrote:
LDAP may be a better data[base|sink] for configuration data then SQL. It may be harder to work with (only because less people have experience with it), and it may be less usefull to a lot of sites (everyone has MySQL installed, not so for LDAP), but preference storage is a common use for LDAP. For sites who already have LDAP deployed, you can leverage the existing authentication data. LDAP stores "objects", SQL "rows"...
Let me take a few moments to describe what I'm looking for, or at least how I'm viewing the system. This is an ideal, and we'll see how close we can get.
Just as we don't tie Mailman to a specific MTA, I'd like to not tie Mailman3 to a specific database/persistence technology. It should be possible to back MM3 against LDAP, MySQL, PostgreSQL, BerkeleyDB, or some combination depending on existing infrastructure. MM3 will come out of the box with /something/ because I still strongly believe in that philosophy. But it should be possible to say, replace the IAuthentication implementation with one that links users to your corporate LDAP database.
This is why we need solid interfaces defining what information MM3 wants to store and retrieve, how to play nicely with transactions, and how to define data sources (and their options). We need to figure out the right breakdown of storage requirements.
The hope then is that we can define these interfaces, and then the MySQL experts can go and implement their version of it, as can the BerkeleyDB experts, the ZODB experts, and the LDAP experts.
-Barry
On Sat, 2004-01-31 at 00:35, Chuq Von Rospach wrote:
I've got a bit of MySQL background. My time is still limited, so I won't commit to coding I can't depend on myself to finish -- but if you want someone to help out on DB design and stuff, I'm in. I'll do what I can and maybe help avoid some of the potholes. I'd like to see this, too.
I think the most critical thing for Mailman 3 will be designing the interfaces between the delivery pipelines and the persistent storages. I won't get into a lot of detail in this thread, but it's getting very close to the time to start talking about the designs I've been working on.
Email isn't always the best way to discuss these things, which is why I'm really hoping we can get some people in a room, with a white board, and hash a bunch of things out. I'd like you all to come of course, but I understand that it likely won't be possible. Don't worry, stuff won't be set in stone until they're discussed here, but if you can get away for a couple of days, it would be awesome.
-Barry
participants (6)
-
Barry Warsaw -
Chuq Von Rospach -
Dale Newfield -
Jeff Warnica -
Kevin McCann -
moron