[IPython-dev] IPython parallel client

[Nathan Faggian]

Hi Doug,

Nice idea! I initially thought a notebook or perhaps an additional tab in
the notebook would be a good way for these features to be introduced. Maybe
that could be down the line with some support.  For now I am focused on
some basic command line tools.

I think once I get a good test framework going it will be easy enough to
prototype a few additional ideas and notebooks are a good way of doing
that.

I think the next step for me is to write a layer of indirection over the
client code so that I can have a common base for GUI and command line
interfaces (keen to make ipcluster_cli).

At the moment the watcher tool is *very* basic but that will improve as I
have time. I would appreciate any help, I think the IPython parallel
framework is excellent!

Cheers,

Nathan.

On 21 April 2014 22:33, Doug Blank <doug.blank at gmail.com> wrote:

> Nathan, thanks for making this, and sharing!
>
> Would it be possible to make this type of code run in a notebook itself? I
> know the GUI allows listing, selection of items, and possibly selection of
> items for additional detail/control. But it might be a nice use-case to see
> if you make the new widgets do what you want, and if not, what is missing.
>
> I'm looking forward to trying this out (and perhaps helping), especially
> once we get our 3rd party kernel working with the parallel/distributed
> interface.
>
> -Doug
>
>
> On Sat, Apr 19, 2014 at 11:46 PM, Nathan Faggian <nathan.faggian at gmail.com
> > wrote:
>
>> Hi,
>>
>> I have started a small (spare time) project called "ipcluster_tools",
>> which is available here:
>>          https://github.com/nfaggian/ipcluster_tools
>>
>> Using the client connection it is pretty easy to poll the state of the
>> cluster thanks to the advice from MinRk. Using this knowledge I built a
>> *very* basic tool to visualise the state of jobs but I would really like to
>> try and write some tests now.
>>
>> Any pointers would be great -- so far I have just used a notebook to
>> inject jobs and used the graphical interface to spin up workers. Since I
>> prefer pytest I would like to try and make something like a "funcarg" that
>> spins up a server (prefer function scope) but the setup and tear-down may
>> be expensive:
>>
>>      http://pytest.org/latest/funcargs.html
>>
>> When I get the tests going to a reasonable level I will change the way
>> that the watcher works. The goal is to (if there is a backend that stores
>> information) be able to introspect metadata also.
>>
>> If anyone is keen to help out please fork the repository!
>>
>> Cheers,
>>
>> Nathan
>>
>> On 16 April 2014 12:18, Nathan Faggian <nathan.faggian at gmail.com> wrote:
>>
>>> Hi MinRk,
>>>
>>> Thanks for the quick response, that is going to be a great help!
>>>
>>> Cheers,
>>>
>>> Nathan.
>>>
>>> On 16 April 2014 12:01, MinRK <benjaminrk at gmail.com> wrote:
>>>
>>>> The method you are probably interested in is
>>>> `Client.queue_status(verbose=True)`, which lets you see at what jobs are
>>>> queued, running, etc. and where. If you want to resubmit a job, you would
>>>> use the `resubmit` method, or abort a queued job with `abort`. Do note that
>>>> killing isn't supported yet in IPython, though you can probably set this up
>>>> fairly easily (trivially, if all your engines are local), as described here:
>>>>
>>>> http://mail.scipy.org/pipermail/ipython-dev/2014-March/013426.html
>>>>
>>>>
>>>>
>>>> On Tue, Apr 15, 2014 at 6:51 PM, Nathan Faggian <
>>>> nathan.faggian at gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am interested in building an ipengine status tool for IPython using
>>>>> Qt. I was wondering if I could get some direction on querying the HUB to
>>>>> get a list of jobs, I have read the "iopubwatcher" example from MinRk,
>>>>> which seems like a good starting point.
>>>>>
>>>>> Though, I am not sure if I should be using the interface provided by
>>>>> the client for this task because it seems like two clients don't share
>>>>> metadata - one client doesn't know what the other has submitted (I may
>>>>> easily be wrong here).
>>>>>
>>>>> Hope my question isn't too vague -- I am interested in being able to
>>>>> point and click on jobs and perform some simple things like kill/re-submit
>>>>> them - it would be really useful!
>>>>>
>>>>> Thanks in advance for any advice,
>>>>>
>>>>> Nathan.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140422/219b30e1/attachment.html>