[Flask] Data Logger with flask

[David Baumgold]

I can’t help you with the process of getting data from the energy meter to a database, but for getting the data from the database to the client and streaming updates, you may want to look into server-sent events instead of polling. I’ve written a Flask extension to make server-sent events easy: it’s called Flask-SSE. https://github.com/singingwolfboy/flask-sse Please let me know if you decide to use it, and how it could be improved!

David Baumgold

On April 19, 2016 at 8:01:05 AM, Anthony Ford (ford.anthonyj at gmail.com) wrote:

To chime in, Andrea has the right idea.

You want to separate your background logger and your web view. If you leverage your SQLAlchemy models right, you can share those in a common package to both applications, or if your SQL is straight-forward, you can skip it and write your own queries.

I have a project for work that is this exact same architecture. For continual logging of multiple parameters of the Arecibo Observatory's Cryogenics system, my background process uses PyModbus to poll the industrial microcontroller modules for the parameters, dumps them in a Postgres DB, and my web front-end pulls the DB data and generates live graphs and JSON structures for the API.

I originally used Supervisor (http://supervisord.org/) to manage the background process, but I've started to explore using uwsgi to launch it as a worker process. No real progress on that yet.

For managing the Polling, I use APScheduler (http://apscheduler.readthedocs.org/en/latest/userguide.html), but that's cause originally I had a more dynamic polling (adjusting rates throughout the day to account for cooler nights and hotter days). I now poll  every 5 minutes, and could have easily done it with a loop.

Good Luck!

Anthony Ford,
KF5IBN,
ford.anthonyj at gmail.com

On Tue, Apr 19, 2016 at 3:19 AM, Andrea D'Amore <and.damore at gmail.com> wrote:
On 18 April 2016 at 15:11, Andreas Dorfner
<andreas.dorfner at stud.hs-regensburg.de> wrote:
> Next, I would like to save that data in a database and create a website to edit the informations as a graph or a diagram.
> For that, I will use flask, but I'm not sure if flask is the right framework for this project?

It depends on what you mean for "right", Flask is certainly capable of
doing that.
Actually any microframework would be, but with Flask you have a nice
ecosystem if you want to add anything later on.
I suggest go with Flask, but this is a Flask mailing list so my advice
may be biased.

> Is it possible to run the webserver for the website and the Pymodbus code in the same Python-File or is it a problem because of the while infinite loop? Maybe it's better to split it to two separate files, one for read data and save it in a database; the second one for the webserver. But how can I link that two files, because I think they have to run simultaneously.

You're doing active polling there, while a webserver adheres to a
client-server paradigm where the client initiates the action while the
server waits.
Those two are different in nature, I've never checked werkzeug's
internals but I'm not sure it won't keep the thread busy. I'd keep
those two parts separated.
Running two programs at the same time isn't an issue, in a sh-like
shell you run:

    ./fetcher.py & ./webserver.py &

or any variation of that, according to your spawner.
You're likely wanting to display the data from the meter as a system
service, i.e. you switch on the beagleboard, you wait an adequate
amount of time and then the system should be working.
How to implement that depends on the operating system that the BB is running.

> Maybe I can use some parts of that tutorial and save the data of the energy meter in the database I have initiazized there?

The app looks fairly simple from what you described, you want a single
view, in which you load a bunch of data from the db and then pass it
to the template for rendering. I'd use one of the nice js plotting
library out there to plot the data passed.

Then you either reload the whole page or you could go with a more
refined solution, implement a specific method returning just the
updated data and then just reload the chart via AJAX rather than the
whole page.. It may not be worth the effort it in this case since the
10 seconds refresh period is quite long and you're not likely to have
more than a couple clients at a time.


--
Andrea
_______________________________________________
Flask mailing list
Flask at python.org
https://mail.python.org/mailman/listinfo/flask

_______________________________________________  
Flask mailing list  
Flask at python.org  
https://mail.python.org/mailman/listinfo/flask  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/flask/attachments/20160419/dae3576a/attachment-0001.html>