[Flask] Flask execute system command / wait / return result ?

[Piotr Grabowski]

Hello again Dear People,

after getting some helpful responses and doing online research, I concluded
that a combination of Celery and Redis will help me create the
scheduling/resource managing pipeline for running the tools by the online
users. However, I am still a little uncertain about one part of the
solution.

When the tools finishes work, it creates output files (3 of them, could be
zipped into 1 though). How do I route back those files uniquely to the user
? The web app will be asking for email and the idea is to just send back
download URLs to given users that their work is finished. However, I have
no idea how to set it up. Do you have any ideas where to look ?

The tool works from the command line so a user clicks submit on the page ->
the job gets added to Redis by Flask -> Celery takes it and throws at an
idle worker -> the script finishes with files generated... and what then ?



Any advice is appreciated!


Best regards,
Piotr

On Wed, Sep 23, 2015 at 1:09 AM, David Nieder <davidnieder at gmx.de> wrote:

> On 22.09.2015 18:38, Piotr Grabowski wrote:
>
>> Hi!
>>
>> I am in a phase of planning a web application for my group (academic
>> research) and would love to get some advice or input from more advanced
>> Flask users/devs.
>>
>> We developed a bioinformatics tool that we run from a command line, it's
>> run simply as ./tool_binary <input> <parameters>, so pretty standard
>> stuff.
>>
>> We also are trying to turn this into an extremely easy-to-use tool for
>> biologists and are looking for a proper online framework. We decided that
>> Flask could be a great solution to keep the whole framework running.
>> (already made it work on Galaxy, but we want to keep it ultra-simple and
>> nice-looking)
>>
>> However, does anyone know if Flask could handle an execution of such
>> command line tool, wait for the tool to finish and send back the user the
>> generated resulting file ?
>>
>>
>> Best regards,
>> Piotr
>>
>>
> Hi Piotr!
>
> Flask doesn't provide a way to do this kind of stuff innately (afaik).
> Basically, you will have to run your program in a second process, take
> the results and send them in an http response. However, without knowing
> how your program works I can't tell you the best way to go about this.
>
> I think running the program and leaving the request hanging until it has
> terminated isn't a good solution. But if you are sure the process will
> only take a very short time this is probably fine.
>
> You could start the process, return an id and fetch the results later
> with a second request.
>
> A solution I think is very nice, is to stream the results back to the
> client. There is a chapter in the docs abouth this:
> http://flask.pocoo.org/docs/0.10/patterns/streaming/
>
> Here is a simple but working example that invokes the ping utility and
> streams it's output back to the client.
>
>
> from shelljob import proc
> from flask import Flask, Response
>
> app = Flask(__name__)
>
> @app.route('/')
> def index():
>     group = proc.Group()
>     group.run(['ping', '-c', '10', '8.8.8.8'])
>
>     def generator():
>         while group.is_pending():
>             data = group.readlines()
>             for handle, lines in data:
>                 yield lines
>
>     return Response(generator(), mimetype='text/plain')
>
>
> if __name__ == '__main__':
>     app.run(debug=True)
>
>
> The shelljob package uses the subprocess module. You could use that
> module directly but I think this is more convenient. As you see, it
> doesn't require much code to do sth like this.
>
> Also: You probably want to let your users set some arguments to your
> program through the web interface. So don't forget to sanitize your
> inputs!
>
> I hope this was of any help
> David
>
>
> _______________________________________________
> Flask mailing list
> Flask at python.org
> https://mail.python.org/mailman/listinfo/flask
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/flask/attachments/20150924/a9fa1738/attachment.html>