[IPython-dev] IPython parallel client

[Doug Blank]

Nathan, thanks for making this, and sharing!

Would it be possible to make this type of code run in a notebook itself? I
know the GUI allows listing, selection of items, and possibly selection of
items for additional detail/control. But it might be a nice use-case to see
if you make the new widgets do what you want, and if not, what is missing.

I'm looking forward to trying this out (and perhaps helping), especially
once we get our 3rd party kernel working with the parallel/distributed
interface.

-Doug


On Sat, Apr 19, 2014 at 11:46 PM, Nathan Faggian
<nathan.faggian at gmail.com>wrote:

> Hi,
>
> I have started a small (spare time) project called "ipcluster_tools",
> which is available here:
>          https://github.com/nfaggian/ipcluster_tools
>
> Using the client connection it is pretty easy to poll the state of the
> cluster thanks to the advice from MinRk. Using this knowledge I built a
> *very* basic tool to visualise the state of jobs but I would really like to
> try and write some tests now.
>
> Any pointers would be great -- so far I have just used a notebook to
> inject jobs and used the graphical interface to spin up workers. Since I
> prefer pytest I would like to try and make something like a "funcarg" that
> spins up a server (prefer function scope) but the setup and tear-down may
> be expensive:
>
>      http://pytest.org/latest/funcargs.html
>
> When I get the tests going to a reasonable level I will change the way
> that the watcher works. The goal is to (if there is a backend that stores
> information) be able to introspect metadata also.
>
> If anyone is keen to help out please fork the repository!
>
> Cheers,
>
> Nathan
>
> On 16 April 2014 12:18, Nathan Faggian <nathan.faggian at gmail.com> wrote:
>
>> Hi MinRk,
>>
>> Thanks for the quick response, that is going to be a great help!
>>
>> Cheers,
>>
>> Nathan.
>>
>> On 16 April 2014 12:01, MinRK <benjaminrk at gmail.com> wrote:
>>
>>> The method you are probably interested in is
>>> `Client.queue_status(verbose=True)`, which lets you see at what jobs are
>>> queued, running, etc. and where. If you want to resubmit a job, you would
>>> use the `resubmit` method, or abort a queued job with `abort`. Do note that
>>> killing isn't supported yet in IPython, though you can probably set this up
>>> fairly easily (trivially, if all your engines are local), as described here:
>>>
>>> http://mail.scipy.org/pipermail/ipython-dev/2014-March/013426.html
>>>
>>>
>>>
>>> On Tue, Apr 15, 2014 at 6:51 PM, Nathan Faggian <
>>> nathan.faggian at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am interested in building an ipengine status tool for IPython using
>>>> Qt. I was wondering if I could get some direction on querying the HUB to
>>>> get a list of jobs, I have read the "iopubwatcher" example from MinRk,
>>>> which seems like a good starting point.
>>>>
>>>> Though, I am not sure if I should be using the interface provided by
>>>> the client for this task because it seems like two clients don't share
>>>> metadata - one client doesn't know what the other has submitted (I may
>>>> easily be wrong here).
>>>>
>>>> Hope my question isn't too vague -- I am interested in being able to
>>>> point and click on jobs and perform some simple things like kill/re-submit
>>>> them - it would be really useful!
>>>>
>>>> Thanks in advance for any advice,
>>>>
>>>> Nathan.
>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140421/45e12e54/attachment.html>