[Mailman-Developers] NNTP Archive - final report

[Alexander Sulfrian]

Hi,

reaching the end of google summer of code, here is my final report. I
have pushed all code to my bzr repository and have a running demo of
the NNTP archive under mailman3.animux.de port 119 and 563. (The
postorius web interface under mailman3.animux.de is not working
completly correctly since I merged the mailman trunk back into my
code)


Features that are working:

 * NNTP Daemon could be started via the runner interface. It could be
   listen on two ports. The first one is a plain tcp connection
   supporting STARTTLS and the second allows nntps
   connections. (Please notice, that thunderbird currently does only
   support nntps connections and does not ask if it should accept the
   certificate. To test the nntps connections you first have to add
   the CACert root certificate to the list of trusted certificate
   authorities.)

 * The nntp daemon supports all mandatory commands and most of the
   commands of the READER capability. Additional to that it supports
   the AUTHINFO extension. The nntp daemon authenticates the user
   against the user information stored by mailman core. Because it has
   only access to the crypt passwords, it could only support
   authentiaction mechanism that uses plain text passwords. Therefor
   it supports and requires the STARTTLS extension (or the plain SSL
   connection) for authentication.

 * The nntp daemon provides access to all public list archives without
   authentication. If a user is logged in, the user has also access to
   its subscribed mailing lists, that has private archives. Currently
   the nntp archive uses a slightly patched version of kittystore (to
   allow access to mails based on its id in the mailing list). There
   is also a module that fills the kittystore if sending a mail via
   the lists. I currently imported some messages from an mbox into
   bla at mailman3.animux.de and feed some real messages into
   test at mailman3.animux.de.


Although gsoc is over now, I would like to continue to work for
mailman3. Next steps for the NNTP archive and archives in general
could be:

 * Currently the nntp archive uses a slightly patches version of
   kittystore (allowing to access mails based on their database
   id). The IArchiver implementation uses the SqlAlchemy interface to
   feed data into the store. It would be much nicer not require a
   second ORM mapper. On solution could be to convert the SqlAlchemy
   interface to Storm models. But I think, before that the whole
   message store infrastructure should be reconsidered. Currently it
   works but their might be the same problems like with the user
   information mentioned earlier on this mailing list. kittystore
   currently stores a superset of all properties all archives
   need. Maybe here (in contrast to the user information) it would be
   a good idea to split it into private information of the archives
   and simply commonly stored mails. (I wrote some thoughts about that
   on my blog[1].)

 * Because of the current architecture of the kittystore the NNTP
   archive could not search for message ids over all known
   mailing lists. This is a feature of the commands ARTICLE, BODY,
   HEAD and STAT. I do not know if there are many news clients out
   there supporting it, but it should be supported in any way.

 * The NNTP protocols requires multiline responses to several
   commands. Currently the responses are build as a list and then send
   as whole to the client. For some commands it would be better to use
   a generator and to steam the line as they are generated. Even now
   (with only a few test lists) I noticed that the querying of the
   database about the count of the articles in the different lists
   slow down the LIST command. Streaming the lines it would be much
   convenient for the user.

 * (optional) Currently it is not possible to POST articles to a
   mailing list via the NNTP archive. It was stated that most of the
   news clients could send a mail as answer. Currently that is okay,
   but I think that it would be not so hard to implement it later as
   addition.

For reference: the current code is available at launchpad[2]. I am
open for improvements and as I said, although gsoc is over now, I
would like to continue to work with mailman. (even though I do not
like bazaar as much as git)


Thanks,
Alex


1: http://blog.animux.de/message-stores-and-mailman-archivers.html
2: https://code.launchpad.net/~alexander-sd4c/mailman/mailman
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/mailman-developers/attachments/20120821/d0ddf621/attachment.pgp>