Queue peek?

[Steve Holden]

Veloz wrote:
> On Mar 3, 1:14 am, Gregory Ewing <greg.ew... at canterbury.ac.nz> wrote:
>> MRAB wrote:
>>> I suppose it depends on the complexity of the data structure. A dict's
>>> methods are threadsafe, for example, but if you have a data structure
>>> where access leads to multiple method calls then collectively they need
>>> a lock.
>> It also depends on the nature of the objects being used
>> as dict keys. Dict operations invoke the hashing and
>> comparison methods of the keys, which in general can
>> execute arbitrary Python code.
>>
>> If the keys are elementary built-in types such as
>> strings or ints, then dict operations will *probably*
>> be atomic. But I think I'd use a lock anyway, to be
>> on the safe side.
>>
>> --
>> Greg
> 
> Unless I missed where you guys were going, I think we got off the main
> point. The main question at hand was this: what's the best way (heck,
> any way) to implement a sort of "peek" whereby a number of processes
> can write results to some common "object" and some other process can
> "peek" into this object, looking for specific items they're interested
> in?
> 
> I've tried doing this with a queue, as follows: children all write
> results to queue, each result has an identifier.  Another interested
> party, which wants to know if identifier XXX has been placed in the
> queue, removes all the items, one by one from the queue, "keeps" the
> one matching the identifier (if found) and puts the rest of the items
> back on the queue, so other interested parties can also look through
> it.
> 
> This is not a good solution, but it illustrates what I'm trying to
> achieve..
> 
> I'm looking at multiprocessing.Manager, ctypes, etc, but nothing's
> really jumped out.
> 
> I also tried creating my own list class which uses locks to provide a
> "peek and remove" method, but I don't have a good way to share an
> instance of this object across processes.
> 
> Any thoughts would be welcomed!
> Michael

Sounds to me like you are using one queue when you should be using
several. If a single process wants to distribute information to multiple
clients, have a queue for each client. This makes it easy for each
client to find out if it has work (if it does its queue is non-empty),

Is this helpful, or a red herring?

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC                 http://www.holdenweb.com/
UPCOMING EVENTS:        http://holdenweb.eventbrite.com/