Here's a bit of low-hanging fruit that might speed up Mailman. I
don't know exactly by how much, but it works by starting Python with
the -S option. On my Sparc boxes I've seen 4-8x improvement in the
startup times of the Python executable. I've tested the patch in that
I'm sure it works, but I haven't put it in the python.org operation
system yet. I plan on doing that next.
Note that this only works for the CGI and mail wrappers, not the
bin scripts, but it should still help out a lot.
Let me know if you give this a shot, and whether you see any
improvement or not. Thanks to Jeremy Hylton for the inspiration.
-Barry
-------------------- snip snip --------------------
Index: common.c
===================================================================
RCS file: /projects/cvsroot/mailman/src/common.c,v
retrieving revision 1.18
retrieving revision 1.19
diff -c -r1.18 -r1.19
*** common.c 1999/07/12 20:32:59 1.18
--- common.c 1999/10/29 15:06:43 1.19
***************
*** 196,212 ****
newenv[j] = NULL;
/* Now put together argv. This will contain first the absolute path
! * to the python executable, then the absolute path to the script,
! * then any additional args passed in argv above.
*/
! newargv = (char**)malloc(sizeof(char*) * (argc + 2));
! newargv[0] = python;
! newargv[1] = (char*)malloc(sizeof(char) * (
strlen(scriptdir) +
strlen(script) +
1));
! strcpy(newargv[1], scriptdir);
! strcat(newargv[1], script);
/* now tack on all the rest of the arguments. we can skip argv's
* first two arguments because, for cgi-wrapper there is only argv[0].
--- 196,215 ----
newenv[j] = NULL;
/* Now put together argv. This will contain first the absolute path
! * to the Python executable, then the -S option (to speed executable
! * start times), then the absolute path to the script, then any
! * additional args passed in argv above.
*/
! newargv = (char**)malloc(sizeof(char*) * (argc + 3));
! j = 0;
! newargv[j++] = python;
! newargv[j++] = "-S";
! newargv[j] = (char*)malloc(sizeof(char) * (
strlen(scriptdir) +
strlen(script) +
1));
! strcpy(newargv[j], scriptdir);
! strcat(newargv[j], script);
/* now tack on all the rest of the arguments. we can skip argv's
* first two arguments because, for cgi-wrapper there is only argv[0].
***************
*** 218,227 ****
*
* TBD: have to make sure this works with alias-wrapper.
*/
! for (i = 2; i < argc; i++)
! newargv[i] = argv[i];
! newargv[i] = NULL;
/* return always means failure */
(void)execve(python, &newargv[0], &newenv[0]);
--- 221,230 ----
*
* TBD: have to make sure this works with alias-wrapper.
*/
! for (i=2, j++; i < argc; i++)
! newargv[j++] = argv[i];
! newargv[j] = NULL;
/* return always means failure */
(void)execve(python, &newargv[0], &newenv[0]);
Hi,
I'm attempting to install mailman 1.0 on a Mac OS X server and managed to
install Python (1.5.2) without too many problems and I ran the ./configure
script.
When I get to the the "make install" stage for mailman it starts chugging along merrily until we get to the following sequence
>for f in answer_majordomo_mail mailcmd mailowner post driver; \
>do \
> /usr/bin/install -c -m 644 $f /Local/Applications/mailman/scripts; \
>done
>cc -c -I. -DPREFIX="\"/Local/Applications/mailman\"" -DPYTHON="\"/usr/local/bin/python\"" -DHELPFUL -g -O2 -DHAVE_STRERROR=1 -DHAVE_SETREGID=1 -DHAVE_SYSLOG=1 -DSTDC_HEADERS=1 -DHAVE_SYSLOG_H=1 -DGETGROUPS_T=gid_t -DHAVE_VSNPRINTF=1 ./common.c
>./common.c:26: `PREFIX' undeclared here (not in a function)
>./common.c:26: `scripts' undeclared here (not in a function)
>./common.c:26: parse error before `;'
>make[1]: *** [common.o] Error 1
>make: *** [install] Error 2
Well I had a look at common.c and there are these lines which appear to cause
the problem
#define SCRIPTDIR PREFIX ## "/scripts/" /* trailing slash */
#define MODULEDIR PREFIX /* no trailing slash */
const char* scriptdir = SCRIPTDIR; [note this line is line 26]
PREFIX does not appear to be an environment variable and it is not explicitly set in common.h
I know this is kind of scratchy stuff but any clues on what to do from here?
Greetings,
Thomas
PS. I'm no guru so assume complete stupidity on my part in any explanation :-)
I am trying to edit my Archives Index Page with the web interface, and my
changes aren't showing up. The archive is private, and works just fine. I
just finished configuring ht://dig to index the private archive and now I
want to put the search doodads on the archive page.
Changes I make on the web form show up in ~/lists/listname/archives.html
but never make it to ~/archives/private/listname/index.html, even after
someone sends mail to the list. The index.html file is overwritten
everytime a message is posted, which blows away anything I tried to add
there.
Just to see if the web-based editor worked, I was able to successfully
edit the General list information page.
I would appreciate any help in modifying this file.
(Mailman 1.0 on a RedHat 6.0 system.)
Thanks!
--
Jerry Adlersfluegel
The problem proved to be with a particular message which could not be
approved,rejected or discarded. I looked at the message and the sender
seemed to be a little odd. So, I made the change below to ListAdmin.py and
it seemed to work okay, I don't know if this might cause other problems.
Adrian
160c160
< 'sender' : msg.GetSender(),
---
> 'sender' : strquote(msg.GetSender()),
The webpage mentions CVS access to sources, but not a CVSROOT.
Does anyone have this string?
--------------------------------------------
Adrian Eyre <mailto:a.eyre@optichrome.com>
Optichrome Computer Solutions Ltd
Maybury Road, Woking, Surrey, GU21 5HX, UK
Tel: +44 1483 740 233 Fax: +44 1483 760 644
http://www.optichrome.com
--------------------------------------------
Howdy to all!
We are running four lists with Mailman 1.0, Python ver.1.5.1, Linux Red Hat,
5.?, and Sendmail, 8.9.3
We installed Mailman late in August.
We have upgraded the smtplib.py to the current version.
We are still sending duplicates of every post, and sometimes up to three or
four copies.
We have searched the archives, and think we understand the problem, but we
are lacking clear direction to resolve it. Can anyone help?
Will Leaver
Joe Whittaker
MASLOM, Inc.
hey there,
two things: (a) is mailman better than majordomo? I assume it must
be right, or you wouldn't be putting all this effort into it. I'm using
majordomo now and I keep meaning to look for something that has better spam
filters. Forgive the stupid questions, but should I switch?
(b) JC has told me that you are considering maybe using BitKeeper for mailman
development. I had a call from a large company today and they sent in their
requirements and I spent a bunch of time answering them. It might be worth
it for you to skim through the questions and answers, I suspect there is some
overlap.
I'd love it if it turned out that BK was useful to you (and I'd be
equally bummed to find out it didn't work for you; if that's the case I
want to know why: if there are technical reasons for you not using it,
those will almost certainly be fixed, you guys have similar issues to
everyone else so that means if it doesn't work for you, it doesn't work
for very many other people).
Finally, if you have questions, you can mail to me and/or dev(a)bitmover.com
. The dev alias is all the people hacking on BK right now.
Cheers,
--lm
-----Forwarded message-----
Hi folks, here are BitMover's responses to your requirements. We think
BitKeeper might be a good fit for you, let us know what you think.
> Open Discussion - Requirements
> a.. Direct FTP access
We provide SSH (secure remote shell), RSH (BSD remote shell), and SMTP
(email) as transports for moving stuff around. We could do a FTP transport
but it seems questionable given that ssh is faster and more secure. If the
issue is Windows, no problem, we support ssh on Windows. Use it all the
time, in fact, between Windows and Unix.
Oh, I forgot, we also work in local and networked file systems (NFS, SMB)
and do resyncs between Unix and Windows using SMB.
> b.. Graphical view of the file tree -or- dynamic tree structure
We actually don't have this right now. But see the checkin response.
We have it on Windows because we integrate with their GUI glop, but
that's not much help.
If we get far enough along and this is a go/no go thing for you, and you're
buying 10,000 seats (:-) then we'll write one. Seriously, it's on the list.
> c.. Check-in and check-out facilities
Got 'em. Both command line and graphical. The command line stuff is like so
bk new file - check a file into the system for the first time
- aliases: bk delta -i, bk ci -i, bk admin -i
bk edit file - check out a file locked for modification
- aliases bk co -l, bk get -e
bk ci file - check in modifications to a file
- aliases: bk delta
You can do all of the above on the entire tree, a sub tree, an explict list:
bk -r new - checks in everything it finds
bk -r edit - locks everything
bk -r ci -yComment - checks in everything
But the best tool is citool, which finds everything you've modified,
lists in a graphical tool, and shows you the diffs as you're doing the
check in. See http://www.bitkeeper.com/citool.html
> d.. Open ended client support allowing developers to use any editing
> tool - FTP to local machine with check-out - edit with whatever tool you
> wish - FTP upload to server with check-in
That's exactly the right answer. Use any editor you want. What you get with
BK is the revision history files locally. If you've worked with RCS or SCCS,
in many respects, BK is like that with all the added stuff you need to have
multiple copies of the revision history. So every playpen is a repository,
you can lock, edit, checkin, browse, etc., everything locally. No need for
network connectivity except when you want to update your tree or the other
tree.
Oh, and you can update "sideways". One developer can slurp in the
changes in a another developer's tree directly, without going through
the "master" or "shared" tree. In CVS terms, I don't have to go back
to the master repository, I can talk directly to you. The hierarchical
nature of the system is strictly a convention thing, you can resync from
anywhere to anywhere.
> e.. Ability to cluster files into "Change Sets" for version control and
> configuration management
Chuckle. We have that in spades. We don't let you move data between playpens
unless you've put it in a changeset (yeah, you could get the diffs and apply
them with patch, we can't stop you, but if you want the system to do it,
you have to "commit" whatever you have done to a changeset).
The normal way you do this is when you pop into citool, there will be
N+1 entries, where N are the files with mods, and the Nth+1 entry is
the ChangeSet file. If you comment the ChangeSet file, you are creating
a ChangeSet. The "diffs" shown for the ChangeSet file are the comments
you have just typed in on all the other files like so
src/foo.c
Fix a bug in arg processing
src/foo.h
Include getopt.h
with the idea being that the ChangeSet comments should be the idea or
concept you just did, while the per file comments are implementation
details.
> f.. Allow user selectable update reports on one or more branches of the
> development system
I think this is what we call a LOD, but I need clarification. What exactly
do you want?
> g.. Ability for CGI to be accessible and scriptable from a remote shell -
> should support Mac/PC/Linux/Unix/etc.
Is this a source management system issue? I don't understand. And one huge
bummer - we don't support the Mac; the system depends pretty heavily on
various Unix tools (sed, sh, tr, diff, etc), which we provide for Windows
but haven't worked up the ompf to do on the Mac. If the Mac is a requirement,
we can support MacOS X no problem, that's Unix, but MacOS 8 is a real drag.
Let me know, that might be a show stopper. There are ways we can work
around this, using appletalk - you could do the source management crud on
Linux and the editing on a Mac. Definitely a hack but I can relate to
the requirement - the Mac has a bunch of useful tools for Web folks.
Long term we will do a port. But that isn't likely to happen in the next
few months.
> h.. Change Logs
You get these for free, it's the ChangeSet comment history.
> i.. Concurrent Development of files - requires intelligent merging agent
> for coordinating updates from multiple developers (CVS does this now)
We do this better than anyone. Period. Everyone else screws up your
history. Here's how: you have two developers A and B. They both do CVS
update and have the same tree. They both modify file foo.c. A checks
in first, so B "lost the race" and has to merge. What CVS will check in
is not B's changes, it's B's changes plus whatever B had to do to merge.
That is **two events** collapsed into one. Why is this a big deal? Undo.
Suppose B's stuff was good and A's stuff wasn't. We can reconstruct
B's stuff without the merge. CVS can't. It lost the information.
Look at http://www.bitkeeper.com/sccstool.html - what you are seeing is the
race and the merge. I'm "lm" and "awc" is my Windows guy. We were working
parallel and the graph that you are seeing is the revision history after
we merged. 1.89 is the merge delta, 1.86.1.1 and 1.86.1.2 are my deltas.
If I want to reconstruct just my work, I do an undo and tell it which
branch I want gone.
And we do all this automagically. That tree of changes is a straight line,
no branches tree as far as the user is concerned. The branches are created
automatically when changes are merged in, you never have to do that.
> j.. Access Control for each user of the development environment: Per user
> basis and Per file basis
We don't do this because this is an operating system issue. How you achieve
the same effect is to restrict access using standard Unix file permissions
on the master repository.
> k.. Provide secure remote access to environment either through secure CRT,
> SSL, domain restrictions, etc.
SSH.
> l.. Ability to differentiate gear, products and templates for exclusive
> use (for one private label) and public use (all private labels)
I don't understand this requirement.
> Roles and Responsibilities
> Engineering
> a.. Priviledged access (on a user by user basis) to everything on the site
> (cgi, kernels, html, graphics, etc.)
This done through the OS. If you have an account and you can read and
write the files, then you can read and write the files.
> b.. Utilize a versioning source control system (SCS) for updates
We provide something which is file format compatible with SCCS, AT&T's
original revision control system from the 70's. A lot of people question
this, there are claims that the SCCS file format is worse than RCS.
That's not true, in fact, the opposite is true. RCS has one potential
advantage that they don't even use: they store the most recent delta
as a clear file and all the previous ones as diffs (backward diffs
for going up the trunk and forward diffs for going down the branches).
If RCS stored an offset and a size in the file, then they could seek to
to the clear text and write the file out in one system call. They don't
do that so they end up reading the whole file anyway (or most of it, it
depends). Whatever. Someone could modify RCS to do what I said and it
would still suck. Here's why:
. No checksum. SCCS checksums the file and verifies the checksum every
time you get the file. BK adds an additional per delta checksum.
The point is that you put your IP into a system; that system should
guard that IP very carefully. RCS makes a performance vs integrity
tradeoff which will end up screwin you in the long run.
. Annotation. SCCS can trivially give you a copy of the file with each
line prefixed by any combination of the revision/user/date which added
that line. In addition, BK can check out a copy of the file with
every line in every revision in the file. Think about that. You
know that somebody changed something and you know the string they
changed but you don't know when. In BK you can find out by doing this
bk sccscat -mu foo.c | grep string
. ChangeSets. SCCS is a changeset engine, people just don't know it. What
that means is that I can edit a file on one branch and say "I also want
that delta over there which is on a different branch". SCCS will happily
include it. You can creat new deltas and include/exclude any arbitrary
list of deltas. It may not be obvious, but this is really cool and has
far reaching ramifications. Not only can RCS not do this, if you tried
to kludge it in, it would grow the file linearly for each include. In
SCCS, including or excluding a delta costs you about 4 bytes per included
delta. In other words, you can construct multiple different views of
your data and it doesn't cost you. Ping me in the conference call about
this if I did a poor job explaining it, it is profoundly important. It's
what makes SCCS a ChangeSet engine and what makes RCS not a ChangeSet
engine.
> c.. Each engineer works in an independant workspace (currently known as
> playpens)
Yup. Each engineer can have as many playpens as s/he wants. Each are
fully independent and get this: NO NO NO environment variables. You work
in a playpen by saying
cd ~/playpen/src
bk vi foo.c
If you want to work in a different playpen, you just say
cd ~/different_playpen/src
bk vi bar.c
Environment variables suck. Just say no. Dare to break the environment
variable habit :-)
> d.. Depending on the engineer's privileges - able to push edits to staging
> server at the branch or global levels
If they have a login and write permissions on the staging area, they can.
I really felt no need to reinvent Unix file permissions. Yeah, this
screws the NT people but they can emulate the same thing by limiting access
to the machine which has the staging area.
> Web Development
> a.. Priviledged access (on a user by user basis) to limited content on the
> site (html, graphics, etc.)
> b.. Utilize a versioning source control system (SCS) for updates
> c.. Each developer works in an independant workspace (currently known as
> playpens)
> d.. Depending on the developers privileges - able to push edits to staging
> server at the branch or global levels
Same as above.
> Quality Assurance
> a.. Utilizes a selective build configuration management system (CMS) that
> allows Q/A to selectivey test and build portions of the site
We're weak here in some regards. We don't support partial repositories.
In other words, when you do a resync, you get the whole tree. To deal
with this, you will naturally split your project up into chunks. Each
of these chunks is a repository. So far so good, but the one bummer is
when you want to share data between two repositories. We currently don't
have any way for data to be in two different "chunks" (we call 'em projects)
at the same time.
This needs more explanation so please ping me during the conference call.
> b.. Ability to rollback change sets and individual files
Chuckle. You bet. The easiest (but somewhat slow) way to roll back is
like so: suppose you want to roll back to ChangeSet 1.123 (which was
conviently tagged with alpha2).
$ bk resync -r..alpha2 master alhpa2-test
That will create a repository which is identical to the master repository
when alpha2 went in (by the way, the tag is just for clarity, anywhere you
use a tag you can use a revision).
The other way is suppose you had a tree with ... alpha2 - 1.124 - 1.125
and you wanted alpha2. You can dothis
$ bk undo 1.124,1.125
and that will DESTROY those two changesets. You'd better have a copy of
them somewhere if you want 'em back).
> c.. Tie in builds with bug tracking system to complete a closed loop
> tracking process
Busted. We want to do this but this is a 2.0 or perhaps 3.0 before we
have it. We have a plan for doing it but right now we don't have diddly.
> d.. Test and submit builds for Go Live!
???
> Release Management
> a.. Schedules and approves builds for go live
> b.. Coordinates builds with Q/A for testing
Seems OK.
> Private Label Partners
> a.. Restricted access to their branch
> b.. Access control based on a case by case basis
I don't know what these are.
---------------------------------------------
One thing you forgot to ask about is file renames. This is another place
where people fall down. We don't (surprise!). We handle file pathname
changes identically to file content changes. Pathnames are revisioned and
propogate. Consider a couple of test cases:
You Me Resync you to me
mv foo bar nothing moves foo to bar in my tree
change foo mv foo bar applies change in foo to bar in my tree
mv foo bar mv foo blech prompts you with name conflict
This is not a big deal until you need it to work but then it is a huge deal.
It can bring your develpment to a halt for days while your engineers
unscramble the mess. We just make that problem go away.
You also forgot file permissions. We pick up the permissions as of the
time that the file was originally checked in "bk new". After that, if you
want to change them, you just say "bk chmod 755 foo.sh" and it saves the
modes. On windows, only the top bits (owner permissions) are used, but
it doesn't stomp on the lower bits.
> Possible Vendors
> a.. Continuus - Continuus http://www.continuus.com/
> b.. Interwoven - Teamsite http://www.interwoven.com/
> c.. Clear Case - Need Vendor Name
Rational Software, http://www.rational.com
> d.. Bit Keeper - Need Vendor Name
BitMover, Inc. And it is "BitKeeper" if you want to be picky.
http://www.bitkeeper.com
Also consider the following:
CVS (it's free and I can show you what you could do to sort of have some
of these features, not all, but some).
Perforce, http://www.perforce.com - I know Chris, great guy, nice little
tool. It doesn't do what you want but it is quite popular and you should
know it exists.
TrueChange from TrueSoft, http://www.truesoft.com - this is the only
commercially available ChangeSet engine other than BitKeeper.
Cheers,
Larry McVoy
President, BitMover, Inc.
-----End of forwarded message-----
--
---
Larry McVoy lm(a)bitmover.com http://www.bitmover.com/lm
On Wed, 13 Oct 1999 23:26:53 -0400 (EDT)
Barry A Warsaw <bwarsaw(a)cnri.reston.va.us> wrote:
<<Regarding Aegis>>
>>>>>> "JCL" == J C Lawrence <claw(a)varesearch.com> writes:
JCL> There's been some brief discussion on Linux-Kernel (searchable
JCL> archives at http://www.mail-archive.com/), but not a whole lot.
JCL> I've never seen a real installation or user report from other
JCL> than the authors.
> And yet it appears to have been out since 1991. Interesting.
That's, umm, one way to put it.
--
J C Lawrence Home: claw(a)kanga.nu
---------(*) Work: claw(a)varesearch.com
... Beware of cromagnons wearing chewing gum and palm pilots ...
On Wed, 13 Oct 1999 18:45:50 -0400 (EDT)
Barry A Warsaw <bwarsaw(a)cnri.reston.va.us> wrote:
> BTW, anybody have real-world experience with Aegis? It's GPL'd
> but I've never heard of it before...
There's been some brief discussion on Linux-Kernel (searchable
archives at http://www.mail-archive.com/), but not a whole lot.
I've never seen a real installation or user report from other than
the authors.
--
J C Lawrence Home: claw(a)kanga.nu
---------(*) Work: claw(a)varesearch.com
... Beware of cromagnons wearing chewing gum and palm pilots ...
On Wed, 13 Oct 1999 18:40:53 -0400 (EDT)
Barry A Warsaw <bwarsaw(a)cnri.reston.va.us> wrote:
> Thanks for the info JC. The BK web pages seem pretty out of date
> though, and I didn't see anything about downloads. Interestingly
> enough I also didn't see any links to the supposedly public free
> software change logs :)
BK is not released *yet*. Access is currently restricted to people
who sign the beta agreement etc. BK should be released fairly soon
tho (Larry: What's the current predict?)
> Everything else sounds interesting. I'd like to come up with a
> solution that would eventually translate to all the projects we've
> got here (JPython and Python) so I want to investigate Aegis, and
> get Guido's feedback on things as well.
Please note that I'm explicitly biased here. I've used and don't
like CVS. I'm aware of Aegis only intellectually. Aegis follows a
somewhat similar design model to BK, but with a much more
overweaning and dictatorial (you will do things our way or you will
not do anything so NYAH!) implementation. Compared to the various
SCM tools I've used at HP, IBM, SGI, etc I'm damn near in love with
BK, so no brownie points for guessing which I'd recommend.
> I appreciate all the feedback, information and experience you guys
> share.
Absolutely. Some of the things I particularly like about BK:
-- Cross platform. It is identical under Linux, Solaris, IRIX,
Windows NT, etc.
-- Resolves 95% of the three way merge problem.
-- The GUI tools for merges (when they do happen) does almost
exactly what you want a three-way merge tool to do.
Some comments I wrote offlist comparing CVS and BK:
-- BK removes 95% of the merge problem. That's enough alone to
sell it to me. BK's merge tools are not perfect, but they're a
sight better than anything else I've seen.
-- BK gives you real audit trails for changes. CVS doesn't. No
more CVSBLAME.
-- BK handles file deletes and renames. CVS doesn't.
-- BK allows you to stage work and to distribute only
known-working source sets. CVS doesn't.
-- Nothing in BK runs as root. CVS' pserver must run as root to
allow UID changes, and pserver was not built with security in mind.
-- BK can be run over SSH. CVS can run pserver connections over
an SSH port forward, but its a pain (I've done it). CVS over SSH
performance also sucks. BK over SSH is quite nippy.
-- BK can use email, even PGP'ed email, as it transport. CVS
can't.
-- BK handles binary files transparently. CVS doesn't even
pretend to. While BK's handling is not great (it checks in the
UUencoded copy), it does work.
-- Nothing happens to a BK repository that you don't explicitly
Okay, and EVERYTHING can be rolled back to its prior state. Got a
bum changeset in? No problem: Just back it out. There are few
controls on what happens to a CVS repository other than user/passwd
authentication, and absolutely no rollback capabilities.
-- One of Larry's big items: BK is very very bandwidth light. You
can easily (and often quickly) sync two massive repositories over a
slow link. Basically under BK only the absolutely needed changes
are sent, nothing else. CVS is not so generous.
-- BK imposes no user heirachy. All BK work areas are first class
repositories. Resync your work area from a repository and the
result is bit identical to the other repository and anybody else can
resync from you. CVS imposes a single central server/many-clients
model. BK allows any sort of tree, or even mesh, to be used.
-- BK handles branches fairly well, sorta. I'm still waiting for
Larry's promised LinesOfDevelopment code so I can get a handle on
what he's trying to do there. If it is what I understand it will be
clean and fairly neat. CVS OTOH is just painful once you have
anything more than a single branch.
-- BK makes it very easy to seperate your work on a source tree
into waht I call "projects" (eg a fix for bug X, new feature Y,
etc), each distinct from the others and then lets you track and
propagate them individually. While similar is possible under CVS,
it is nowhere near as easy and the individual propagation just ain't
there.
--
J C Lawrence Home: claw(a)kanga.nu
---------(*) Work: claw(a)varesearch.com
... Beware of cromagnons wearing chewing gum and palm pilots ...
