[Python-bugs-list] [ python-Bugs-731644 ] GIL not released around getaddrinfo()

[SourceForge.net]

Bugs item #731644, was opened at 2003-05-03 01:23
Message generated for change (Comment added) made by jvr
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=731644&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Just van Rossum (jvr)
Assigned to: Nobody/Anonymous (nobody)
Summary: GIL not released around getaddrinfo()

Initial Comment:
I have a situation where sock.connect() blocks in getaddrinfo(), but 
since the GIL isn't released around getaddrinfo() calls, other threads 
also block.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-05-03 20:02

Message:
Logged In: YES 
user_id=92689

I've attached a first cut of a patch. It implements wrappers for getaddrinfo 
and getnameinfo, to take the burden of dealing with the lock and the GIL 
away from the caller. As suggested by MvL in prvt mail, I've simply reused 
the existing gethostbyname lock, but renamed it to netdb_lock.

What needs to be reviewed carefully is when exactly USE_NETDB_LOCK 
(formerly known as USE_GETHOSTBYNAME_LOCK) is defined. It happens to 
be defined for my platform, so it "works for me".

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-05-03 16:00

Message:
Logged In: YES 
user_id=21627

Mark: the original complaint came from BSD users. So if we
only release the GIL on a "good" platform, the BSD users
aren't helped. Likewise, unless compiled by VC7.x, Windows
users are not helped, since Python would use the getaddrinfo
emulation. So if this as attempted, I think we need a
best-effort solution for all systems.

You could look at the Tcl lock. I recommend you look at the
2.2 sources, since the Tcl lock logic got more complicated
recently (as Tcl now also has a multi-threaded mode of
operation).

If your platform is Mac OS X, don't trust that it is too
BSDish in this respect. If both FreeBSD and NeXT had some
function, Apple apparently has always chosen the NeXT
version. getaddrinfo was one such case. But then, it appears
that the NeXT version is even worse than the FreeBSD version.


----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-05-03 15:56

Message:
Logged In: YES 
user_id=92689

(Never mind, I found plenty of examples in sockemodule.c itself.)

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-05-03 15:40

Message:
Logged In: YES 
user_id=92689

Since my platform is BSD-ish, I want a lock ;-)

Yes, I would like to work on a patch, except I will need some hand-holding, 
as I've never done anything with threads from C. Can you point me to 
examples of such a lock in the Python source code?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-05-03 14:30

Message:
Logged In: YES 
user_id=14198

How about simply not releasing the GIL on these platforms? 
Seems simpler than a new lock.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-05-03 14:18

Message:
Logged In: YES 
user_id=21627

See python.org/sf/604210.

Yes, using a separate lock would be possible. At a minimum,
the BSD implementation and the getaddrinfo emulation are not
thread-safe; there may be more. 

As I write in 604210: it would be fine if a separate lock is
used only on the known-bad platforms, since the RFC states
getaddrinfo is thread-safe.

Would you like to work on a patch?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-05-03 12:09

Message:
Logged In: YES 
user_id=92689

Isn't that an orthogonal issue? In that we should then use a lock for 
getaddrinfo(), yet do release the GIL.

I find it unacceptable that my GUI main thread can hang for several seconds 
(or more) just by doing a sock.connect() in a different thread.

Short of that, can we find out on _which_ platforms getaddrinfo() isn't 
thread-safe and work around that?

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-05-03 11:03

Message:
Logged In: YES 
user_id=21627

This is not a bug. getaddrinfo isn't thread-safe on some
systems, so we must protect it from multiple access.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=731644&group_id=5470