We are currently using Mailman 2 for a number of mailing lists on our website (PHP based). We have mailing lists but need to have these people grouped in the same groups on our website and don't want to have to maintain 2 lists of the same people. I'd like to move us to Mailman 3 and take advantage of the REST API and write my own PHP code in interact with it.
All of that being said I'm missing something in getting my basic setup finalized. I have a web server (Apache) running and have installed mailman core. When I try to access the REST API I get some variation of connection refused depending on what I'm using (python command line, curl, etc). I'm feeling like apache doesn't know where to point port 8001, and I don't know what to tell it.
For the previous version of mailman you had to put and Alias and a Script Alias into the apache config. I've seen some documentation about what to put in for Postorius, but haven't found anything for just mailman.
Any thoughts and/or pointers would be greatly appreciated.
-Erin
On 07/11/2017 05:50 PM, Erin Test wrote:
All of that being said I'm missing something in getting my basic setup finalized. I have a web server (Apache) running and have installed mailman core. When I try to access the REST API I get some variation of connection refused depending on what I'm using (python command line, curl, etc). I'm feeling like apache doesn't know where to point port 8001, and I don't know what to tell it.
You really can't make use of the REST API via a web browser or through a normal web server like Apache. Web browsers basically do HTTP GET and can submit forms via HTTP POST. The REST API requires other HTTP methods for full functionality and returns responses that are not particularly human friendly and are definitely not HTML.
You should be able to access the core REST API via curl. See <http://mailman.readthedocs.io/en/latest/src/mailman/rest/docs/rest.html> for more information.
For the previous version of mailman you had to put and Alias and a Script Alias into the apache config. I've seen some documentation about what to put in for Postorius, but haven't found anything for just mailman.
The rest API in Mailman core is not a web UI. That's what Postorius is for.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jul 11, 2017, at 05:50 PM, Erin Test wrote:
All of that being said I'm missing something in getting my basic setup finalized. I have a web server (Apache) running and have installed mailman core. When I try to access the REST API I get some variation of connection refused depending on what I'm using (python command line, curl, etc). I'm feeling like apache doesn't know where to point port 8001, and I don't know what to tell it.
To supplement what Mark said, there are a few things to keep in mind with the REST API. First, you really don't need to put Apache in front of it, as Mailman Core already provides an HTTP server for the REST API [*].
Second, you definitely do *not* want to expose the REST API on the public internet. It's an administrative API so it has full access to you list server. It only provides very basic authentication and no user based authorization, so any process with the basic auth keys can do anything.
We have a *very* nascent project called Lemme which is geared toward a public facing authenticating REST proxy. While we have ideas for how it would work, we haven't had many resources available to work on it. Contributions of course would be greatly welcomed!
https://gitlab.com/mailman/lemme/blob/master/README.rst
Cheers, -Barry
[*] By default we use the stdlib simple wsgi server, which is convenient but makes no claims of being very performant. Hopefully for 3.2 I will switch to aiohttp, uvloop, or some other better performing substrate.
Hi Barry and Mark
Thanks for your responses. I have no desire to open the REST API to the public, I'm just trying to access it at all and assumed I needed to provided a web server, but that's apparently not the case. Other than mailman start is there something I'm missing that is needed to start the Mailman included web server?
To access the REST API I've tried both command line curl calls: curl -I -u restadmin http://localhost:8001/3.1/system/versions Enter host password for user 'restadmin': curl: (7) Failed to connect to localhost port 8001: Connection refused
And calls from within python:
import requests response = requests.get( ... 'http://localhost:8001/3.1/system/versions', ... auth=('restadmin', 'restpass')) Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
... more errors presented after this but I'm assuming they relate to the Connection refused.
Thanks, -Erin
On 07/12/2017 11:57 AM, Erin Test wrote:
To access the REST API I've tried both command line curl calls: curl -I -u restadmin http://localhost:8001/3.1/system/versions Enter host password for user 'restadmin': curl: (7) Failed to connect to localhost port 8001: Connection refused
Is something else listening on port 8001 (like maybe Apache). Do you have iptables or some other firewall and if so have you opened port 8001.
What does
sudo netstat -l -n -t -p
say? Is something listening on 8001 and if so, what?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I'm not aware of anything else using that port.
sudo netstat -l -n -t -p 8001 8001: unknown or uninstrumented protocol
I'm on a mac and I read the options for netstat can be different between OSX/BSD and Linux, so I verified using lsof (8889 is the port mysql is using)
leaves:~ ejustice$ lsof -n -i4TCP:8001 | grep LISTEN leaves:~ ejustice$ lsof -n -i4TCP:8889 | grep LISTEN mysqld 84142 ejustice 17u IPv6 0x7164e83ec7901d4b 0t0 TCP *:ddi-tcp-2 (LISTEN)
Also I've tried changing the default port in /etc/mailman.cfg (from 8001 to 9001 and a couple other randomly chosen ports) and I get the same Connection Refused. When I try to use the port mysql is using 8889 I get
sudo curl -I -u restadmin http://localhost:8889/3.1/system/versions Enter host password for user 'restadmin': curl: (8) Weird server reply
So I don't think it's a conflict with a port that is in use.
-Erin
On Jul 12, 2017, at 12:04 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 07/12/2017 11:57 AM, Erin Test wrote:
To access the REST API I've tried both command line curl calls: curl -I -u restadmin http://localhost:8001/3.1/system/versions Enter host password for user 'restadmin': curl: (7) Failed to connect to localhost port 8001: Connection refused
Is something else listening on port 8001 (like maybe Apache). Do you have iptables or some other firewall and if so have you opened port 8001.
What does
sudo netstat -l -n -t -p
say? Is something listening on 8001 and if so, what?
On 07/12/2017 01:11 PM, Erin Test wrote:
I'm not aware of anything else using that port.
sudo netstat -l -n -t -p 8001 8001: unknown or uninstrumented protocol
sudo netstat -l -n -t -p
no 8001. -p means list the names of the processes listining. The output will be a list of all the processes listening on TCP ports. Look for port 8001 in that list. If it's not there, nothing is listening on 8001. If it is, the name of the process will be listed.
Here's a sample output:
$ sudo netstat -l -n -t -p Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 192.168.5.13:8301 0.0.0.0:* LISTEN 1465/consul
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 24794/nginx
tcp 0 0 127.0.0.1:8400 0.0.0.0:* LISTEN 1465/consul
tcp 0 0 127.0.0.1:8500 0.0.0.0:* LISTEN 1465/consul
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1508/sshd
tcp 0 0 127.0.0.1:8024 0.0.0.0:* LISTEN 1220/python3
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 1796/postgres
tcp 0 0 127.0.0.1:8600 0.0.0.0:* LISTEN 1465/consul
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 1814/master
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 24794/nginx
tcp 0 0 127.0.0.1:8891 0.0.0.0:* LISTEN 1711/opendkim
tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 24368/python
tcp 0 0 127.0.0.1:8001 0.0.0.0:* LISTEN 1221/python3
This is mailman listening on port 8001
tcp6 0 0 :::22 :::* LISTEN 1508/sshd
tcp6 0 0 :::25 :::* LISTEN 1814/master
I'm on a mac and I read the options for netstat can be different between OSX/BSD and Linux, so I verified using lsof (8889 is the port mysql is using)
leaves:~ ejustice$ lsof -n -i4TCP:8001 | grep LISTEN
So maybe nothing is listening on 8001.
leaves:~ ejustice$ lsof -n -i4TCP:8889 | grep LISTEN mysqld 84142 ejustice 17u IPv6 0x7164e83ec7901d4b 0t0 TCP *:ddi-tcp-2 (LISTEN)
Also I've tried changing the default port in /etc/mailman.cfg (from 8001 to 9001 and a couple other randomly chosen ports) and I get the same Connection Refused. When I try to use the port mysql is using 8889 I get
sudo curl -I -u restadmin http://localhost:8889/3.1/system/versions Enter host password for user 'restadmin': curl: (8) Weird server reply
So I don't think it's a conflict with a port that is in use.
The conflict would be if some other process was listening on 8001, Mailman wouldn't be able to.
Have you started mailman (mailman start)?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jul 12, 2017, at 1:34 PM, Mark Sapiro <mark@msapiro.net> wrote:
Have you started mailman (mailman start)?
I thought I did, but when I tried to stop it just now it said it wasn't running. And when I try it now I'm seeing sudo mailman start Password: Starting Mailman's master runner /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python: can't open file '/sbin/master': [Errno 2] No such file or directory
Thanks, -Erin
On 07/12/2017 01:42 PM, Erin Test wrote:
I thought I did, but when I tried to stop it just now it said it wasn't running. And when I try it now I'm seeing sudo mailman start Password: Starting Mailman's master runner /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python: can't open file '/sbin/master': [Errno 2] No such file or directory
How did you install Mailman?
You might find this thread interesting <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/5TPWA3T47SFX35GDHJUIOPT7XVBXHW4C/>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jul 12, 2017, at 3:49 PM, Mark Sapiro <mark@msapiro.net> wrote: You might find this thread interesting <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/5... <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/5TPWA3T47SFX35GDHJUIOPT7XVBXHW4C/>>.
That link pointed me in the right direction. It was the following change I'd added to the mailman.cfg.
[mailman] # Which paths.* filesystem layout to use. layout: fhs
while following the instructions here: http://docs.mailman3.org/en/latest/prodsetup.html?highlight=apache#setting-u... <http://docs.mailman3.org/en/latest/prodsetup.html?highlight=apache#setting-up-mailman-core>
Once I commented that line out I'm able to follow the examples https://mailman.readthedocs.io/en/latest/src/mailman/rest/docs/basic.html <https://mailman.readthedocs.io/en/latest/src/mailman/rest/docs/basic.html> without issue.
Thank you Mark and Barry!
-Erin
Hi Erin,
On Jul 12, 2017, at 11:57 AM, Erin Test wrote:
To access the REST API I've tried both command line curl calls: curl -I -u restadmin http://localhost:8001/3.1/system/versions Enter host password for user 'restadmin': curl: (7) Failed to connect to localhost port 8001: Connection refused
What does mailman info say? That will print the hostname:port that the REST
server is listening on, and the basic auth credentials in use. All of those
are of course configurable in your mailman.cfg file.
If your curl arguments match what mailman info says, then there must be
something else going on. Maybe there's a firewall issue? You can also check
the Mailman logs to see if you're getting some exception or other in the
server, and of course you can also adjust the logging level in the mailman.cfg
file.
Cheers, -Barry
Everything matches between mailman info and what I'm putting into the curl call. Also mailman info & mailman conf are showing changes when I make them so I know /etc/mailman.cfg file is being read.
As to the logs I've got the logging level set at debug for all the logging categories but I'm not seeing anything in the log files.
-Erin
On Jul 12, 2017, at 12:10 PM, Barry Warsaw <barry@list.org> wrote:
Hi Erin,
On Jul 12, 2017, at 11:57 AM, Erin Test wrote:
To access the REST API I've tried both command line curl calls: curl -I -u restadmin http://localhost:8001/3.1/system/versions Enter host password for user 'restadmin': curl: (7) Failed to connect to localhost port 8001: Connection refused
What does
mailman infosay? That will print the hostname:port that the REST server is listening on, and the basic auth credentials in use. All of those are of course configurable in your mailman.cfg file.If your curl arguments match what
mailman infosays, then there must be something else going on. Maybe there's a firewall issue? You can also check the Mailman logs to see if you're getting some exception or other in the server, and of course you can also adjust the logging level in the mailman.cfg file.Cheers, -Barry
participants (3)
-
Barry Warsaw -
Erin Test -
Mark Sapiro