Coverage not catching execution in my Flask app
I have a simple little Flask app and created a suitably simple test script, aimed at making sure I covered much of the executable code. It fires up the app with coverage enabled and hits it with a bunch of requests, then exits. When testing, I have coverage enabled: CMD="$(which flask) run -h localhost -p $PORT" if [ "x$DOCOVER" = "xtrue" ] ; then coverage run -a $CMD else $CMD fi Coverage seems to do the right thing, creating a .coverage file on exit, and when I run the annotate, report, or html commands it does the right thing. Still, it never actually marks any statements within functions as having been executed. Here's coverage from a manual run where I poked the server with three debug endpoints:
if FLASK_DEBUG: @app.route("/env") def printenv(): ! return jsonify(dict(os.environ))
@app.route('/api/help') def app_help(): """Print available functions."""
! func_list = {} ! for rule in app.url_map.iter_rules(): ! if rule.endpoint != 'static': ! func_list[rule.rule] = str(app.view_functions[rule.endpoint]) ! return jsonify(func_list)
@app.get('/shutdown') def shutdown():
! func = request.environ.get('werkzeug.server.shutdown') ! if func is None: ! raise RuntimeError('Not running with the Werkzeug Server') ! func() ! return 'Server shutting down...\n' Clearly, coverage noticed the action which happened during import, but despite the fact that I really did request /env, /api/help, and /shutdown, it never thought any of them were called. Any idea what I'm missing? Thx, Skip Montanaro
Hi, During development, web frameworks often spawn a subprocess to serve requests so that they can reload the code if it changes. You might be measuring only the first process, and not the spawned process that is actually doing the work. --Ned. On 1/22/22 10:42 AM, Skip Montanaro wrote:
I have a simple little Flask app and created a suitably simple test script, aimed at making sure I covered much of the executable code. It fires up the app with coverage enabled and hits it with a bunch of requests, then exits. When testing, I have coverage enabled:
CMD="$(which flask) run -h localhost -p $PORT" if [ "x$DOCOVER" = "xtrue" ] ; then coverage run -a $CMD else $CMD fi
Coverage seems to do the right thing, creating a .coverage file on exit, and when I run the annotate, report, or html commands it does the right thing. Still, it never actually marks any statements within functions as having been executed. Here's coverage from a manual run where I poked the server with three debug endpoints:
if FLASK_DEBUG: @app.route("/env") def printenv(): ! return jsonify(dict(os.environ))
@app.route('/api/help') def app_help(): """Print available functions.""" ! func_list = {} ! for rule in app.url_map.iter_rules(): ! if rule.endpoint != 'static': ! func_list[rule.rule] = str(app.view_functions[rule.endpoint]) ! return jsonify(func_list)
@app.get('/shutdown') def shutdown(): ! func = request.environ.get('werkzeug.server.shutdown') ! if func is None: ! raise RuntimeError('Not running with the Werkzeug Server') ! func() ! return 'Server shutting down...\n'
Clearly, coverage noticed the action which happened during import, but despite the fact that I really did request /env, /api/help, and /shutdown, it never thought any of them were called.
Any idea what I'm missing?
Thx,
Skip Montanaro
_______________________________________________ code-quality mailing list --code-quality@python.org To unsubscribe send an email tocode-quality-leave@python.org https://mail.python.org/mailman3/lists/code-quality.python.org/ Member address:ned@nedbatchelder.com
During development, web frameworks often spawn a subprocess to serve requests so that they can reload the code if it changes. You might be measuring only the first process, and not the spawned process that is actually doing the work.
Ah, thanks Ned. That makes sense. I suspect the subprocess coverage section in the documentation might be helpful. Skip
During development, web frameworks often spawn a subprocess to serve
requests so that they can reload the code if it changes. You might be measuring only the first process, and not the spawned process that is actually doing the work.
Ah, thanks Ned. That makes sense. I suspect the subprocess coverage section in the documentation might be helpful.
I fiddled around with sitecustomize.py, defining COVERAGE_PROCESS_START, and tweaking the --concurrency flag. Nothing worked. In fact, my code coverage went to zero. :-( I then looked at the flask docs to see if it had some clues and noticed its run command has a --no-reload flag. Setting that worked like a charm. Since I'm not actively fiddling on-the-fly when running tests, that's perfect. Now I have nice coverage stats. I figured I'd toss out my solution in case someone else comes along looking for flask+coverage assistance. Skip
participants (2)
-
Ned Batchelder -
Skip Montanaro