Small change to REST API for held subscription requests

I want to make some minor backward incompatible changes to the JSON representation for pending mailing lists requests, e.g. subscription holds. You'll see these for example in urls like:
>>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/requests')
entry 0:
delivery_mode: regular
display_name: Anne Person
email: anne@example.com
http_etag: "..."
language: en
request_id: ...
type: subscription
when: 2005-08-01T07:49:23
The first change is that type: subscription
requests will change their
address
key to email
for consistency with other parts of the API. Second,
there will be no password
key.
Let me know if this is a problem, and I can copy email
to address
so that
both would appear, and add a dummy password
key.
Cheers, -Barry

Fine with me.
as
On 22 Mar 2015, at 12:32 pm, Barry Warsaw <barry@list.org> wrote:
I want to make some minor backward incompatible changes to the JSON representation for pending mailing lists requests, e.g. subscription holds. You'll see these for example in urls like:
The first change is that type: subscription
requests will change their
address
key to email
for consistency with other parts of the API. Second,
there will be no password
key.
Let me know if this is a problem, and I can copy email
to address
so that
both would appear, and add a dummy password
key.
Cheers, -Barry
Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40s...
Security Policy: http://wiki.list.org/x/QIA9

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 22.03.2015 um 02:32 schrieb Barry Warsaw:
As for the password
key: It's recognized by mailman.client and
exposed within the mlist instance's requests
property. But it's used
nowhere in postorius and I would be surprised if it's used anywhere in
hyperkitty (Aurélien...?). I guess it can go away.
As for the email/address keys: We can also just keep the address
key
in mailman.client and let it use the same value as email
. That way
we would still have a fallback but the REST API would stay clean and
wouldn't send any redundant information over the wire.
Can you notify me when your API changes are pushed to LP, so I can update mailman.client accordingly? Just to keep the window as small as possible where we have non-compatible states in the mailman/mailman.client trunks. Especially during GSoC application week...
Florian
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQEcBAEBAgAGBQJVDyxKAAoJEEceGbPdavl7VicIAIvPKbwYfpAHDHBDNHjQALjZ eTjz1KjaH/q9cyAQ1rmhttH1emiz5BukKOIvUTxeANc8ooscHXmUN+Prs1ahgebe tSS7lKvc14kx9QW1HRvbQdnQpYcicNyTSZhTjeD/+RDIaUE+MWaRRxGbcOkHXCHT OrmXeBSOAYGHPrNsExUcIBwRz8IdjFajQ8FIYS/GddsFeotUBJfQxQ7xdkDJMYkB 6iuPtuWwEKXvbxZLMgPtpCpuxYxOeS+cfQarXCdXLmrwruYNDPK0T16TQOa+QC+o /3oNR5aKOTWmy3Dg+Jf2+CJ2eKmQ/NPqvWotXkl2h5CdwWu+TB8RZBTA3XQ91Z4= =v1VH -----END PGP SIGNATURE-----

On Mar 22, 2015, at 09:55 PM, Florian Fuchs wrote:
TBH, I desperately want to purge user passwords from core too! I think they're almost completely unused; the only reason to keep them would be to help Postorious or other REST clients.
Cool.
Will do. I'm working on the subscription policy branch again, and this is a useful refactoring I'm committing before other work. I'm trying to decide whether to commit this to trunk now or when subpolicy lands. Stay tuned!
Cheers, -Barry

How would user passwords work if not stored in the core?
When someone logs in currently I just ask the core if the password is valid and off we go.
as
On 23 Mar 2015, at 8:19 am, Barry Warsaw <barry@list.org> wrote:
On Mar 22, 2015, at 09:55 PM, Florian Fuchs wrote:
TBH, I desperately want to purge user passwords from core too! I think they're almost completely unused; the only reason to keep them would be to help Postorious or other REST clients.
Cool.
Will do. I'm working on the subscription policy branch again, and this is a useful refactoring I'm committing before other work. I'm trying to decide whether to commit this to trunk now or when subpolicy lands. Stay tuned!
Cheers, -Barry
Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40s...
Security Policy: http://wiki.list.org/x/QIA9

Any thoughts on how we could integrate with LDAP?
I’d be happy to authenticate against LDAP but that leaves the open question of how to synchronize the Mailman user database with the LDAP server.
Maybe of Mailman had some sort of event notification hook system for when users add/edit/delete. Or if there was some sort of generalised message bus within the system that funnels messages about changes to the user database. Something sort of LDAP synch process could then use those to synchronise an LDAP database. An event hook or message queue would allow changes to the users to be immediately pushed into LDAP.
Maybe the other possibility is some sort of direct integration such that Mailman, if given details of an LDAP server, can directly make changes to the LDAP system each time a user is added/edited/deleted.
There was some discussion about LDAP integration here: https://mail.python.org/pipermail/mailman-developers/2015-February/024412.ht...
LDAP seems like a pretty configuration heavy thing to put in to a standlone Mailman system. I’m currently researching to find some sort of minimla/lightweight LDAP integration approach.
as
On 23 Mar 2015, at 9:46 am, Barry Warsaw <barry@list.org> wrote:
On Mar 23, 2015, at 09:34 AM, Andrew Stuart wrote:
WFM. As I said, the core doesn't (currently) use it, but certainly it's available through the REST API, so if REST clients need it, we'll keep it.
-Barry

Andrew Stuart writes:
IMHO, what we really want (besides a pony, do you raise ponies?) is "enterprise" integration, by which I mean that Mailman doesn't do auth at all, but provides something like OAuth with a bundled lightweight backend that does password auth, and bundled adapters that can connect to social auth (OpenID, Mozilla Persona) or an enterprise's LDAP directory.
Sounds like a great GSoC project to me!!

On Mar 23, 2015, at 08:39 PM, Andrew Stuart wrote:
Any thoughts on how we could integrate with LDAP?
The intent is that the user database could be backed by, or augmented by LDAP, although that is currently a goal, not reality.
Internally, Mailman does use zope.events to signal things nonlocally to other parts of the system. For example, events get fired when the configuration changes, or when a message gets accepted. Some of those events are connected to handlers, but others are just waiting to be used by some future plugin or other. It's cheap to add events and handlers.
For users though we don't have much atm. Just a PasswordChangeEvent and an unsubscribe event. There's no reason why we couldn't add more to help with external user database synchronization.
The other point is that the use of interfaces and the Zope Component Architecture (ZCA) is supposed to provide a layer of abstraction that could be used as plug points for integrating with other backends.
Think of the IUserManager. The existing implementation of that interface is mailman.model.usermanager.UserManager, and it stores user information in the local SQL database. This connection between interface and implementation is through src/mailman/config/configure.zcml. Since there's only one user manager in the system, it's defined as a utility:
<utility provides="mailman.interfaces.usermanager.IUserManager" factory="mailman.model.usermanager.UserManager" />
Thus when the code calls getUtility(IUserManager) and gets back the built-in UserManager object, after that, everything works through the interface. You could conceivably create a plugin with your own LDAP backed UserManager, change the utility connection, and then the rest of Mailman would just keep on working.
In theory of course. No one to my knowledge has actually tried to explore these ideas.
Cheers, -Barry

Fine with me.
as
On 22 Mar 2015, at 12:32 pm, Barry Warsaw <barry@list.org> wrote:
I want to make some minor backward incompatible changes to the JSON representation for pending mailing lists requests, e.g. subscription holds. You'll see these for example in urls like:
The first change is that type: subscription
requests will change their
address
key to email
for consistency with other parts of the API. Second,
there will be no password
key.
Let me know if this is a problem, and I can copy email
to address
so that
both would appear, and add a dummy password
key.
Cheers, -Barry
Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40s...
Security Policy: http://wiki.list.org/x/QIA9

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 22.03.2015 um 02:32 schrieb Barry Warsaw:
As for the password
key: It's recognized by mailman.client and
exposed within the mlist instance's requests
property. But it's used
nowhere in postorius and I would be surprised if it's used anywhere in
hyperkitty (Aurélien...?). I guess it can go away.
As for the email/address keys: We can also just keep the address
key
in mailman.client and let it use the same value as email
. That way
we would still have a fallback but the REST API would stay clean and
wouldn't send any redundant information over the wire.
Can you notify me when your API changes are pushed to LP, so I can update mailman.client accordingly? Just to keep the window as small as possible where we have non-compatible states in the mailman/mailman.client trunks. Especially during GSoC application week...
Florian
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQEcBAEBAgAGBQJVDyxKAAoJEEceGbPdavl7VicIAIvPKbwYfpAHDHBDNHjQALjZ eTjz1KjaH/q9cyAQ1rmhttH1emiz5BukKOIvUTxeANc8ooscHXmUN+Prs1ahgebe tSS7lKvc14kx9QW1HRvbQdnQpYcicNyTSZhTjeD/+RDIaUE+MWaRRxGbcOkHXCHT OrmXeBSOAYGHPrNsExUcIBwRz8IdjFajQ8FIYS/GddsFeotUBJfQxQ7xdkDJMYkB 6iuPtuWwEKXvbxZLMgPtpCpuxYxOeS+cfQarXCdXLmrwruYNDPK0T16TQOa+QC+o /3oNR5aKOTWmy3Dg+Jf2+CJ2eKmQ/NPqvWotXkl2h5CdwWu+TB8RZBTA3XQ91Z4= =v1VH -----END PGP SIGNATURE-----

On Mar 22, 2015, at 09:55 PM, Florian Fuchs wrote:
TBH, I desperately want to purge user passwords from core too! I think they're almost completely unused; the only reason to keep them would be to help Postorious or other REST clients.
Cool.
Will do. I'm working on the subscription policy branch again, and this is a useful refactoring I'm committing before other work. I'm trying to decide whether to commit this to trunk now or when subpolicy lands. Stay tuned!
Cheers, -Barry

How would user passwords work if not stored in the core?
When someone logs in currently I just ask the core if the password is valid and off we go.
as
On 23 Mar 2015, at 8:19 am, Barry Warsaw <barry@list.org> wrote:
On Mar 22, 2015, at 09:55 PM, Florian Fuchs wrote:
TBH, I desperately want to purge user passwords from core too! I think they're almost completely unused; the only reason to keep them would be to help Postorious or other REST clients.
Cool.
Will do. I'm working on the subscription policy branch again, and this is a useful refactoring I'm committing before other work. I'm trying to decide whether to commit this to trunk now or when subpolicy lands. Stay tuned!
Cheers, -Barry
Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40s...
Security Policy: http://wiki.list.org/x/QIA9

Any thoughts on how we could integrate with LDAP?
I’d be happy to authenticate against LDAP but that leaves the open question of how to synchronize the Mailman user database with the LDAP server.
Maybe of Mailman had some sort of event notification hook system for when users add/edit/delete. Or if there was some sort of generalised message bus within the system that funnels messages about changes to the user database. Something sort of LDAP synch process could then use those to synchronise an LDAP database. An event hook or message queue would allow changes to the users to be immediately pushed into LDAP.
Maybe the other possibility is some sort of direct integration such that Mailman, if given details of an LDAP server, can directly make changes to the LDAP system each time a user is added/edited/deleted.
There was some discussion about LDAP integration here: https://mail.python.org/pipermail/mailman-developers/2015-February/024412.ht...
LDAP seems like a pretty configuration heavy thing to put in to a standlone Mailman system. I’m currently researching to find some sort of minimla/lightweight LDAP integration approach.
as
On 23 Mar 2015, at 9:46 am, Barry Warsaw <barry@list.org> wrote:
On Mar 23, 2015, at 09:34 AM, Andrew Stuart wrote:
WFM. As I said, the core doesn't (currently) use it, but certainly it's available through the REST API, so if REST clients need it, we'll keep it.
-Barry

Andrew Stuart writes:
IMHO, what we really want (besides a pony, do you raise ponies?) is "enterprise" integration, by which I mean that Mailman doesn't do auth at all, but provides something like OAuth with a bundled lightweight backend that does password auth, and bundled adapters that can connect to social auth (OpenID, Mozilla Persona) or an enterprise's LDAP directory.
Sounds like a great GSoC project to me!!

On Mar 23, 2015, at 08:39 PM, Andrew Stuart wrote:
Any thoughts on how we could integrate with LDAP?
The intent is that the user database could be backed by, or augmented by LDAP, although that is currently a goal, not reality.
Internally, Mailman does use zope.events to signal things nonlocally to other parts of the system. For example, events get fired when the configuration changes, or when a message gets accepted. Some of those events are connected to handlers, but others are just waiting to be used by some future plugin or other. It's cheap to add events and handlers.
For users though we don't have much atm. Just a PasswordChangeEvent and an unsubscribe event. There's no reason why we couldn't add more to help with external user database synchronization.
The other point is that the use of interfaces and the Zope Component Architecture (ZCA) is supposed to provide a layer of abstraction that could be used as plug points for integrating with other backends.
Think of the IUserManager. The existing implementation of that interface is mailman.model.usermanager.UserManager, and it stores user information in the local SQL database. This connection between interface and implementation is through src/mailman/config/configure.zcml. Since there's only one user manager in the system, it's defined as a utility:
<utility provides="mailman.interfaces.usermanager.IUserManager" factory="mailman.model.usermanager.UserManager" />
Thus when the code calls getUtility(IUserManager) and gets back the built-in UserManager object, after that, everything works through the interface. You could conceivably create a plugin with your own LDAP backed UserManager, change the utility connection, and then the rest of Mailman would just keep on working.
In theory of course. No one to my knowledge has actually tried to explore these ideas.
Cheers, -Barry
participants (6)
-
Andrew Stuart
-
Aurelien Bompard
-
Barry Warsaw
-
Barry Warsaw
-
Florian Fuchs
-
Stephen J. Turnbull