It’s reproducible for me, I’m not sure what might be different in our configurations.

For a clean start I deleted my Mailman 3 installation. I downloaded the latest version and installed it. I deleted and recreated my databases after first making sure they were empty.

To see the problem I execute the following command, which instructs the REST API to create a new user with email address of mailmanadmin@example.org (you should be able to execute this too, it’s the default REST username and password in the authorization header)

curl -X POST --data "email=mailmanadmin@example.org" --header "authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz" http://localhost:8001/3.0/users

Each time the command above is executed, an additional record is added to preferences, uid and user, despite reporting an “address already exists” error.

I’m definitely not running anything else that would be accessing the REST API.

I tried it on both Postgres and sqlite and got the same problem.

I had a look at src/mailman/rest/users.py to see if I could see a bug but there wasn’t anything obvious that would lead to this happening.

Here is the output of a sqlite dump after running the above curl command three times.

(venv3.4)ubuntu@server01:~/mailman/var/data$ sqlite3 sqlite.db .dump PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE preferences ( id INTEGER NOT NULL, acknowledge_posts BOOLEAN, hide_address BOOLEAN, preferred_language VARCHAR, receive_list_copy BOOLEAN, receive_own_postings BOOLEAN, delivery_mode INTEGER, delivery_status INTEGER, PRIMARY KEY (id), CHECK (acknowledge_posts IN (0, 1)), CHECK (hide_address IN (0, 1)), CHECK (receive_list_copy IN (0, 1)), CHECK (receive_own_postings IN (0, 1)) ); INSERT INTO "preferences" VALUES(1,NULL,NULL,NULL,NULL,NULL,NULL,NULL); INSERT INTO "preferences" VALUES(2,NULL,NULL,NULL,NULL,NULL,NULL,NULL); INSERT INTO "preferences" VALUES(3,NULL,NULL,NULL,NULL,NULL,NULL,NULL); INSERT INTO "preferences" VALUES(4,NULL,NULL,NULL,NULL,NULL,NULL,NULL); CREATE TABLE domain ( id INTEGER NOT NULL, mail_host VARCHAR, base_url VARCHAR, description VARCHAR, contact_address VARCHAR, PRIMARY KEY (id) ); CREATE TABLE mailinglist ( id INTEGER NOT NULL, list_name VARCHAR, mail_host VARCHAR, list_id VARCHAR, allow_list_posts BOOLEAN, include_rfc2369_headers BOOLEAN, advertised BOOLEAN, anonymous_list BOOLEAN, created_at DATETIME, next_request_id INTEGER, next_digest_number INTEGER, digest_last_sent_at DATETIME, volume INTEGER, last_post_at DATETIME, accept_these_nonmembers BLOB, admin_immed_notify BOOLEAN, admin_notify_mchanges BOOLEAN, administrivia BOOLEAN, archive_policy INTEGER, autoresponse_grace_period DATETIME, autorespond_owner INTEGER, autoresponse_owner_text VARCHAR, autorespond_postings INTEGER, autoresponse_postings_text VARCHAR, autorespond_requests INTEGER, autoresponse_request_text VARCHAR, filter_action INTEGER, filter_content BOOLEAN, collapse_alternatives BOOLEAN, convert_html_to_plaintext BOOLEAN, bounce_info_stale_after DATETIME, bounce_matching_headers VARCHAR, bounce_notify_owner_on_disable BOOLEAN, bounce_notify_owner_on_removal BOOLEAN, bounce_score_threshold INTEGER, bounce_you_are_disabled_warnings INTEGER, bounce_you_are_disabled_warnings_interval DATETIME, forward_unrecognized_bounces_to INTEGER, process_bounces BOOLEAN, default_member_action INTEGER, default_nonmember_action INTEGER, description VARCHAR, digest_footer_uri VARCHAR, digest_header_uri VARCHAR, digest_is_default BOOLEAN, digest_send_periodic BOOLEAN, digest_size_threshold FLOAT, digest_volume_frequency INTEGER, digestable BOOLEAN, discard_these_nonmembers BLOB, emergency BOOLEAN, encode_ascii_prefixes BOOLEAN, first_strip_reply_to BOOLEAN, footer_uri VARCHAR, forward_auto_discards BOOLEAN, gateway_to_mail BOOLEAN, gateway_to_news BOOLEAN, goodbye_message_uri VARCHAR, header_matches BLOB, header_uri VARCHAR, hold_these_nonmembers BLOB, info VARCHAR, linked_newsgroup VARCHAR, max_days_to_hold INTEGER, max_message_size INTEGER, max_num_recipients INTEGER, member_moderation_notice VARCHAR, mime_is_default_digest BOOLEAN, moderator_password BLOB, newsgroup_moderation INTEGER, nntp_prefix_subject_too BOOLEAN, nondigestable BOOLEAN, nonmember_rejection_notice VARCHAR, obscure_addresses BOOLEAN, owner_chain VARCHAR, owner_pipeline VARCHAR, personalize INTEGER, post_id INTEGER, posting_chain VARCHAR, posting_pipeline VARCHAR, preferred_language VARCHAR, display_name VARCHAR, reject_these_nonmembers BLOB, reply_goes_to_list INTEGER, reply_to_address VARCHAR, require_explicit_destination BOOLEAN, respond_to_post_requests BOOLEAN, scrub_nondigest BOOLEAN, send_goodbye_message BOOLEAN, send_welcome_message BOOLEAN, subject_prefix VARCHAR, topics BLOB, topics_bodylines_limit INTEGER, topics_enabled BOOLEAN, welcome_message_uri VARCHAR, PRIMARY KEY (id), CHECK (allow_list_posts IN (0, 1)), CHECK (include_rfc2369_headers IN (0, 1)), CHECK (advertised IN (0, 1)), CHECK (anonymous_list IN (0, 1)), CHECK (admin_immed_notify IN (0, 1)), CHECK (admin_notify_mchanges IN (0, 1)), CHECK (administrivia IN (0, 1)), CHECK (filter_content IN (0, 1)), CHECK (collapse_alternatives IN (0, 1)), CHECK (convert_html_to_plaintext IN (0, 1)), CHECK (bounce_notify_owner_on_disable IN (0, 1)), CHECK (bounce_notify_owner_on_removal IN (0, 1)), CHECK (process_bounces IN (0, 1)), CHECK (digest_is_default IN (0, 1)), CHECK (digest_send_periodic IN (0, 1)), CHECK (digestable IN (0, 1)), CHECK (emergency IN (0, 1)), CHECK (encode_ascii_prefixes IN (0, 1)), CHECK (first_strip_reply_to IN (0, 1)), CHECK (forward_auto_discards IN (0, 1)), CHECK (gateway_to_mail IN (0, 1)), CHECK (gateway_to_news IN (0, 1)), CHECK (mime_is_default_digest IN (0, 1)), CHECK (nntp_prefix_subject_too IN (0, 1)), CHECK (nondigestable IN (0, 1)), CHECK (obscure_addresses IN (0, 1)), CHECK (require_explicit_destination IN (0, 1)), CHECK (respond_to_post_requests IN (0, 1)), CHECK (scrub_nondigest IN (0, 1)), CHECK (send_goodbye_message IN (0, 1)), CHECK (send_welcome_message IN (0, 1)), CHECK (topics_enabled IN (0, 1)) ); CREATE TABLE bounceevent ( id INTEGER NOT NULL, list_id VARCHAR, email VARCHAR, timestamp DATETIME, message_id VARCHAR, context INTEGER, processed BOOLEAN, PRIMARY KEY (id), CHECK (processed IN (0, 1)) ); CREATE TABLE ban ( id INTEGER NOT NULL, email VARCHAR, list_id VARCHAR, PRIMARY KEY (id) ); CREATE TABLE pended ( id INTEGER NOT NULL, token VARCHAR, expiration_date DATETIME, PRIMARY KEY (id) ); CREATE TABLE uid ( id INTEGER NOT NULL, uid CHAR(32), PRIMARY KEY (id) ); INSERT INTO "uid" VALUES(1,'aefa4caf73b64729a0245deb9f85f640'); INSERT INTO "uid" VALUES(2,'66a84f1623694e468d375e47d5cb5030'); INSERT INTO "uid" VALUES(3,'1fc2c84e096c41e2ac07a01024415759'); CREATE TABLE message ( id INTEGER NOT NULL, message_id VARCHAR, message_id_hash VARCHAR, path VARCHAR, PRIMARY KEY (id) ); CREATE TABLE user ( id INTEGER NOT NULL, display_name VARCHAR, password VARCHAR, _user_id CHAR(32), _created_on DATETIME, _preferred_address_id INTEGER, preferences_id INTEGER, PRIMARY KEY (id), FOREIGN KEY(preferences_id) REFERENCES preferences (id) ); INSERT INTO "user" VALUES(1,'','$6$rounds=98929$cAg4vQkCh8KYEQYM$sNVQnEWkQc9i3mZiTrafNk5DdEL1N0qD2B1.pV68g8oX5hDvNysiHMzUGTEAcvcmdi4GsQS1OjkmZ9ZjkPyyd/','aefa4caf73b64729a0245deb9f85f640','2015-02-04 10:11:03.080036',NULL,1); INSERT INTO "user" VALUES(2,'',NULL,'66a84f1623694e468d375e47d5cb5030','2015-02-04 10:12:14.014487',NULL,3); INSERT INTO "user" VALUES(3,'',NULL,'1fc2c84e096c41e2ac07a01024415759','2015-02-04 10:13:47.575524',NULL,4); CREATE TABLE pendedkeyvalue ( id INTEGER NOT NULL, "key" VARCHAR, value VARCHAR, pended_id INTEGER, PRIMARY KEY (id), FOREIGN KEY(pended_id) REFERENCES pended (id) ); CREATE TABLE listarchiver ( id INTEGER NOT NULL, mailing_list_id INTEGER NOT NULL, name VARCHAR NOT NULL, _is_enabled BOOLEAN, PRIMARY KEY (id), FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id), CHECK (_is_enabled IN (0, 1)) ); CREATE TABLE acceptablealias ( id INTEGER NOT NULL, mailing_list_id INTEGER NOT NULL, alias VARCHAR NOT NULL, PRIMARY KEY (id), FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id) ); CREATE TABLE contentfilter ( id INTEGER NOT NULL, mailing_list_id INTEGER, filter_type INTEGER, filter_pattern VARCHAR, PRIMARY KEY (id), FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id) ); CREATE TABLE _request ( id INTEGER NOT NULL, "key" VARCHAR, request_type INTEGER, data_hash VARCHAR, mailing_list_id INTEGER, PRIMARY KEY (id), FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id) ); CREATE TABLE address ( id INTEGER NOT NULL, email VARCHAR, _original VARCHAR, display_name VARCHAR, verified_on DATETIME, registered_on DATETIME, user_id INTEGER, preferences_id INTEGER, PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES user (id), FOREIGN KEY(preferences_id) REFERENCES preferences (id) ); INSERT INTO "address" VALUES(1,'mailmanadmin@example.org',NULL,'',NULL,'2015-02-04 10:11:03.108943',1,2); CREATE TABLE member ( id INTEGER NOT NULL, _member_id CHAR(32), role INTEGER, list_id VARCHAR, moderation_action INTEGER, address_id INTEGER, preferences_id INTEGER, user_id INTEGER, PRIMARY KEY (id), FOREIGN KEY(address_id) REFERENCES address (id), FOREIGN KEY(preferences_id) REFERENCES preferences (id), FOREIGN KEY(user_id) REFERENCES user (id) ); CREATE TABLE autoresponserecord ( id INTEGER NOT NULL, address_id INTEGER, mailing_list_id INTEGER, response_type INTEGER, date_sent DATE, PRIMARY KEY (id), FOREIGN KEY(address_id) REFERENCES address (id), FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id) ); CREATE TABLE onelastdigest ( id INTEGER NOT NULL, mailing_list_id INTEGER, address_id INTEGER, delivery_mode INTEGER, PRIMARY KEY (id), FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id), FOREIGN KEY(address_id) REFERENCES address (id) ); CREATE TABLE alembic_version ( version_num VARCHAR(32) NOT NULL ); INSERT INTO "alembic_version" VALUES('33e1f5f6fa8'); CREATE INDEX ix_uid_uid ON uid (uid); CREATE INDEX ix_user__user_id ON user (_user_id); CREATE INDEX ix_user_preferences_id ON user (preferences_id); CREATE INDEX ix_pendedkeyvalue_pended_id ON pendedkeyvalue (pended_id); CREATE INDEX ix_listarchiver_mailing_list_id ON listarchiver (mailing_list_id); CREATE INDEX ix_acceptablealias_alias ON acceptablealias (alias); CREATE INDEX ix_acceptablealias_mailing_list_id ON acceptablealias (mailing_list_id); CREATE INDEX ix_contentfilter_mailing_list_id ON contentfilter (mailing_list_id); CREATE INDEX ix__request_mailing_list_id ON _request (mailing_list_id); CREATE INDEX ix_address_user_id ON address (user_id); CREATE INDEX ix_address_preferences_id ON address (preferences_id); CREATE INDEX ix_autoresponserecord_address_id ON autoresponserecord (address_id); CREATE INDEX ix_autoresponserecord_mailing_list_id ON autoresponserecord (mailing_list_id); COMMIT; (venv3.4)ubuntu@server01:~/mailman/var/data$

On 2 Feb 2015, at 9:28 am, Barry Warsaw <barry@list.org> wrote:

On Feb 01, 2015, at 07:23 PM, Andrew Stuart wrote:

In the commands below you can see that I start by getting a list of users, it returns "total_size": 1

How was this user created and with what email address?

I then execute a POST to the REST API multiple times. Each time it says “Address already exists”, and then getting a list of users says total_size": 2

Is this the way its intended to work or a bug?

Nope, and I can't reproduce it. Are you sure there are no other processes running, and nothing else hitting the REST API that could be racing with that POST?

Try tailing the mailman.log and watching for http activity.

Cheers, -Barry

Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40s...

Security Policy: http://wiki.list.org/x/QIA9

Hi Andrew,

Thanks for the detailed response. I'll try to find some time tonight to go through your recipe step-by-step. Some questions in the meantime:

On Feb 04, 2015, at 09:26 PM, Andrew Stuart wrote:

For a clean start I deleted my Mailman 3 installation. I downloaded the latest version and installed it.

Are you working from the 3.0b5 tarball or the bzr trunk? If the former, are you using the "A" release (Python 2.7) or the "B" release (Python 3.4)?

If you're using the tarball, can you please try again with the head of the bzr trunk?

Cheers, -Barry