[Flask] Flask execute system command / wait / return result ?

David Baumgold david at davidbaumgold.com
Thu Sep 24 15:28:21 CEST 2015


When the script has finished generating the files, they’ll need to be put somewhere that is designed to hold and serve static files. Many people use a service like Amazon S3 for this, but if you don’t have many users, or the files aren’t very large, then you could just store them on the filesystem of your server. At any rate, your worker should be able to also send out an email to the user at the end of the task, with a URL to wherever you’re serving the static files from, either a URL to the file(s) on S3, or a URL to your webserver that identifies the file(s) there. If you’re going to be sending email, you may want to have a look at Flask-Mail (https://pythonhosted.org/Flask-Mail/), or use an email service like Amazon SES to send the email on your behalf.

Another thing to consider is how sensitive these generated files are. The files are generated by a request from a user, so how bad would it be if someone other than the requestor was able to download these files? If it would be no problem at all, then you can use sensible, guessable URLs. If it would be a problem, then you should include some sort of randomly-generated string in the URL name — that makes it almost impossible to guess the URL, but the user still won’t have any problem because the email link will contain the randomly-generated string. If it would be a huge, business-destroying problem, then you should implement authentication on the file download so that the user has to prove that they are the same person who requested the file, in order to download it. Authentication for downloads can be a tricky thing to get right due to performance concerns, and it's usually a bad user experience, so be careful how you do it.

David Baumgold


From: Piotr Grabowski <kajocina at gmail.com>
Reply: Piotr Grabowski <kajocina at gmail.com>>
Date: September 24, 2015 at 9:05:11 AM
To: 
Cc: flask at python.org <flask at python.org>>
Subject:  Re: [Flask] Flask execute system command / wait / return result ?  

Hello again Dear People,

after getting some helpful responses and doing online research, I concluded that a combination of Celery and Redis will help me create the scheduling/resource managing pipeline for running the tools by the online users. However, I am still a little uncertain about one part of the solution.

When the tools finishes work, it creates output files (3 of them, could be zipped into 1 though). How do I route back those files uniquely to the user ? The web app will be asking for email and the idea is to just send back download URLs to given users that their work is finished. However, I have no idea how to set it up. Do you have any ideas where to look ?

The tool works from the command line so a user clicks submit on the page -> the job gets added to Redis by Flask -> Celery takes it and throws at an idle worker -> the script finishes with files generated... and what then ?



Any advice is appreciated!


Best regards,
Piotr

On Wed, Sep 23, 2015 at 1:09 AM, David Nieder <davidnieder at gmx.de> wrote:
On 22.09.2015 18:38, Piotr Grabowski wrote:
Hi!

I am in a phase of planning a web application for my group (academic
research) and would love to get some advice or input from more advanced
Flask users/devs.

We developed a bioinformatics tool that we run from a command line, it's
run simply as ./tool_binary <input> <parameters>, so pretty standard stuff.

We also are trying to turn this into an extremely easy-to-use tool for
biologists and are looking for a proper online framework. We decided that
Flask could be a great solution to keep the whole framework running.
(already made it work on Galaxy, but we want to keep it ultra-simple and
nice-looking)

However, does anyone know if Flask could handle an execution of such
command line tool, wait for the tool to finish and send back the user the
generated resulting file ?


Best regards,
Piotr


Hi Piotr!

Flask doesn't provide a way to do this kind of stuff innately (afaik).
Basically, you will have to run your program in a second process, take
the results and send them in an http response. However, without knowing
how your program works I can't tell you the best way to go about this.

I think running the program and leaving the request hanging until it has
terminated isn't a good solution. But if you are sure the process will
only take a very short time this is probably fine.

You could start the process, return an id and fetch the results later
with a second request.

A solution I think is very nice, is to stream the results back to the
client. There is a chapter in the docs abouth this:
http://flask.pocoo.org/docs/0.10/patterns/streaming/

Here is a simple but working example that invokes the ping utility and
streams it's output back to the client.


from shelljob import proc
from flask import Flask, Response

app = Flask(__name__)

@app.route('/')
def index():
    group = proc.Group()
    group.run(['ping', '-c', '10', '8.8.8.8'])

    def generator():
        while group.is_pending():
            data = group.readlines()
            for handle, lines in data:
                yield lines

    return Response(generator(), mimetype='text/plain')


if __name__ == '__main__':
    app.run(debug=True)


The shelljob package uses the subprocess module. You could use that
module directly but I think this is more convenient. As you see, it
doesn't require much code to do sth like this.

Also: You probably want to let your users set some arguments to your
program through the web interface. So don't forget to sanitize your
inputs!

I hope this was of any help
David


_______________________________________________
Flask mailing list
Flask at python.org
https://mail.python.org/mailman/listinfo/flask

_______________________________________________  
Flask mailing list  
Flask at python.org  
https://mail.python.org/mailman/listinfo/flask  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/flask/attachments/20150924/940634e9/attachment-0001.html>


More information about the Flask mailing list