[Flask] REST based File Sharing server Implementation in Flask

[ub at artfacts.net]

On 25.06.2017 18:43, G.S.Vijay Raajaa wrote:
> Hi UB,
> 
> Thanks for the idea. I shall probe more into the same. But I have quick
> clarifications, even with mongo/gridfs and having the connectionI/O
> delegated to the database driver makes the transaction blocking? Kindly
> correct me if I am wrong. Is there a async way of handling the upload to
> mongo and keep flask serve other incoming request?
well, there will be workers. if you deploy your app in uwsgi or gunicorn,
you set the number of workers in you config. these will then handle
requests (more or less) indepently.
pymongo will have a connection pool to also handle multiple connections. [2]


> On the same note, w.r.t mongodb-gridFS, how do you think the write
> performance is when compared to storing them directly on the filesystem,
> same w.r.t to read for serving?
i guess here's the bad news. it might be slower than writing directly to
the file system, but it might be better when you have app and storage
separated and need to write your files over the network. but you need to
consult benchmarks to be sure.

> I like to idea of scalability when you have file store within DB.
i think here's the upside. also seems, mongodb can help you to store and
organize 8search) file metadata. if that is a concern at all.

[2]
http://api.mongodb.com/python/current/faq.html#how-does-connection-pooling-work-in-pymongo
> 
> Regards,
> Vijay Raajaa GS 
> 
> On Sun, Jun 25, 2017 at 8:08 PM, G.S.Vijay Raajaa
> <gsvijayraajaa at gmail.com <mailto:gsvijayraajaa at gmail.com>> wrote:
> 
>     Hi,
> 
>     I am trying to build a REST based file sharing server on top of
>     flask. I am looking for a scalable stack.
> 
>     The requirement goes as follows;
> 
>     1) Cater multiple file uploads.
>     2) Files can be of the order of 30-100GB. The server hosting the
>     python webserver should not read the file to memory and spill to disk.
>     3) Non blocking - when user uploads multiple files.
>     4) Non blocking - when multiple user try to upload files.
> 
>     Pointers to any existing open source project is also much appreciated.
> 
>     Idea inspired from https://filebin.net/
> 
>     Seeing if celery + Flask can come handy. Kindly suggest the WSGI to
>     use it for production too.
> 
>     Looking forward to your valuable suggestions.
> 
>     Regards,
>     Vijay Raajaa GS 
> 
> 
> 
> 
> _______________________________________________
> Flask mailing list
> Flask at python.org
> https://mail.python.org/mailman/listinfo/flask
>