[Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler

[Tim Peters]

[Guido, on reworking select()]
> Yes, I think we do need this.  PyObject_AsFileDescriptor() calls the
> fileno() method which could be a Python call.

Ya, but for me to believe that *matters*, I have to believe people worried
about high-performance select are passing objects with Python-implemented
fileno methods to begin with.  That's quite a stretch, isn't it?  They have
to pay for fileno() calls regardless, the question is only whether they pay
it once or twice per object.

> Maybe we could use a more sophisticated construct where we use a short
> array allocated on the stack for lists of fewer than 100 (say) files
> but use malloc for larger ones?

Another idea:  screw it <wink>.  We have to allocate three lists anyway to
hold the results.  Allocate them at the start instead of the end, and then
they can do double-duty, holding the cached fileno() results across the
select() call.

Fleshing it out a little,

def list2set(inputlist, fdset):
    "Return (list of file descriptors, biggest fd seen); set fdset slots."
    n = len(inputlist)
    result = [None] * n
    if _MSC_VER:
        biggest = 0
    else:
        biggest = -1
     for i in range(n):
        obj = inputlist[i]
        fd = PyObject_AsFileDescriptor(obj)
        if fd == -1:
            return NULL, -1
        result[i] = obj, fd
        FD_SET(fd, fdset)
        if not _MSC_VER:
            biggest = max(biggest, fd)
    return result, biggest

# iolist is the result of an earlier list2set() call.  It gets shrunk
# to hold just the objects corresponding to the fds in fdset.

def set2list(iolist, fdset):
    lastindex = 0
    for obj, fd in iolist:
        if FD_ISSET(fd, fdset):
            iolist[lastindex] = obj
            lastindex += 1
    if lastindex < len(iolist):
        del iolist[lastindex:]

So it still does the perversely <wink> optimized caching, but no stack space
required, no free() calls, only the 3 malloc() calls that are needed anyway,
and up to 3 realloc() calls to shrink the output lists.  It does create a
bunch of 2-tuples, but they're on a free list and come cheap.