How and where to add tests for Python 3
Hello! I'm new in this mailing, so I will introduce myself first. I'm François (alias Linkid), a French engineering student who code mainly in Python. I'm involved in some open source projects and some of them like Django or OpenStack. I would like to add a test for Python 3 to check an import, but I don't really know on which project (pep8, pyflakes, …) to do that and how to do it (by a push request on github or somewhere else, by a bug report, …). I searched some clue and I think my test should be used by pyflakes, but I'm not sure… So, here are my questions: - on which project should I add a Python 3 import test ? - how I should process ? I think it could help be a lot if someone explain me what one project really check, if it is possible. Thank you. François Magimel (Linkid)
On Sun, Aug 24, 2014 at 3:07 PM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Hello!
I'm new in this mailing, so I will introduce myself first. I'm François (alias Linkid), a French engineering student who code mainly in Python. I'm involved in some open source projects and some of them like Django or OpenStack.
I would like to add a test for Python 3 to check an import, but I don't really know on which project (pep8, pyflakes, …) to do that and how to do it (by a push request on github or somewhere else, by a bug report, …). I searched some clue and I think my test should be used by pyflakes, but I'm not sure… So, here are my questions: - on which project should I add a Python 3 import test ? - how I should process ?
I think it could help be a lot if someone explain me what one project really check, if it is possible.
Thank you.
François Magimel (Linkid)
Hello François, I'm not quite sure anyone on this list can answer your question. You haven't given us any guidance about what the check is that you would like to add. Do you want to check for imports that are not used in Python 3? Are you looking for imports that would fail on Python 3 instead? Are you looking to ensure some style of imports? Perhaps you're looking to do what openstack/hacking already does (ensure only modules are imported)? There are too many possibilities for us to be helpful. Further, unless what you wish to add is part of http://legacy.python.org/dev/peps/pep-0008/, I can at least tell you that it does *not* belong in pep8. If you can give us more details I can also tell you whether it belongs in PyFlakes. If it turns out it does not belong in PyFlakes, you can always write and release your own plugin for Flake8. Sorry I cannot be more helpful, Ian
Le 25/08/2014 02:24, Ian Cordasco a écrit :
On Sun, Aug 24, 2014 at 3:07 PM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Hello!
I'm new in this mailing, so I will introduce myself first. I'm François (alias Linkid), a French engineering student who code mainly in Python. I'm involved in some open source projects and some of them like Django or OpenStack.
I would like to add a test for Python 3 to check an import, but I don't really know on which project (pep8, pyflakes, …) to do that and how to do it (by a push request on github or somewhere else, by a bug report, …). I searched some clue and I think my test should be used by pyflakes, but I'm not sure… So, here are my questions: - on which project should I add a Python 3 import test ? - how I should process ?
I think it could help be a lot if someone explain me what one project really check, if it is possible.
Thank you.
François Magimel (Linkid) Hello François,
I'm not quite sure anyone on this list can answer your question. You haven't given us any guidance about what the check is that you would like to add. Do you want to check for imports that are not used in Python 3? Are you looking for imports that would fail on Python 3 instead? Are you looking to ensure some style of imports? Perhaps you're looking to do what openstack/hacking already does (ensure only modules are imported)? There are too many possibilities for us to be helpful.
Further, unless what you wish to add is part of http://legacy.python.org/dev/peps/pep-0008/, I can at least tell you that it does *not* belong in pep8. If you can give us more details I can also tell you whether it belongs in PyFlakes. If it turns out it does not belong in PyFlakes, you can always write and release your own plugin for Flake8.
Sorry I cannot be more helpful, Ian
Hello, Oh, yes, it will be better if I give some details. My code didn't pass my python 3 tests due to an import of string. As a matter of fact, I was using string.letters which is not in python 3 anymore. However, when I tested my code with flake8, I wasn't warned about it. So, I would like to add this warning (maybe to PyFlakes, but I'm not sure). And I think it is not only an OpenStack issue because it concerns every projects. So, if someone can help me to add it (to PyFlakes?) by giving me the process (PR on GitHub?), I will be very happy to contribute :). Thank you for your answer, Ian. François
So Flake8 would actually recognize that if you installed Flake8 under Python 3 (e.g., pip3 install flake8), and then run flake8 from Python 3. You can do this with a virtualenv more easily like: virtualenv -ppython3 path/to/virtualenv source path/to/virtualenv/bin/activate pip install flake8 flake8 project/ PyFlakes will only run checks for the version of Python it is installed on. Let me know if you run into any problems with this. On Mon, Aug 25, 2014 at 3:36 AM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Le 25/08/2014 02:24, Ian Cordasco a écrit :
On Sun, Aug 24, 2014 at 3:07 PM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Hello!
I'm new in this mailing, so I will introduce myself first. I'm François (alias Linkid), a French engineering student who code mainly in Python. I'm involved in some open source projects and some of them like Django or OpenStack.
I would like to add a test for Python 3 to check an import, but I don't really know on which project (pep8, pyflakes, …) to do that and how to do it (by a push request on github or somewhere else, by a bug report, …). I searched some clue and I think my test should be used by pyflakes, but I'm not sure… So, here are my questions: - on which project should I add a Python 3 import test ? - how I should process ?
I think it could help be a lot if someone explain me what one project really check, if it is possible.
Thank you.
François Magimel (Linkid) Hello François,
I'm not quite sure anyone on this list can answer your question. You haven't given us any guidance about what the check is that you would like to add. Do you want to check for imports that are not used in Python 3? Are you looking for imports that would fail on Python 3 instead? Are you looking to ensure some style of imports? Perhaps you're looking to do what openstack/hacking already does (ensure only modules are imported)? There are too many possibilities for us to be helpful.
Further, unless what you wish to add is part of http://legacy.python.org/dev/peps/pep-0008/, I can at least tell you that it does *not* belong in pep8. If you can give us more details I can also tell you whether it belongs in PyFlakes. If it turns out it does not belong in PyFlakes, you can always write and release your own plugin for Flake8.
Sorry I cannot be more helpful, Ian
Hello,
Oh, yes, it will be better if I give some details. My code didn't pass my python 3 tests due to an import of string. As a matter of fact, I was using string.letters which is not in python 3 anymore. However, when I tested my code with flake8, I wasn't warned about it. So, I would like to add this warning (maybe to PyFlakes, but I'm not sure). And I think it is not only an OpenStack issue because it concerns every projects. So, if someone can help me to add it (to PyFlakes?) by giving me the process (PR on GitHub?), I will be very happy to contribute :).
Thank you for your answer, Ian. François
I tested with python 3.4, but Flake8 didn't recognized this error… But I thought Flake8 only runs pep8, PyFlakes & co. tests, no? So, if I would like to add this test, it may belong to PyFlakes, right? Le 25/08/2014 14:38, Ian Cordasco a écrit :
So Flake8 would actually recognize that if you installed Flake8 under Python 3 (e.g., pip3 install flake8), and then run flake8 from Python 3.
You can do this with a virtualenv more easily like:
virtualenv -ppython3 path/to/virtualenv source path/to/virtualenv/bin/activate pip install flake8 flake8 project/
PyFlakes will only run checks for the version of Python it is installed on. Let me know if you run into any problems with this.
On Mon, Aug 25, 2014 at 3:36 AM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Le 25/08/2014 02:24, Ian Cordasco a écrit :
On Sun, Aug 24, 2014 at 3:07 PM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Hello!
I'm new in this mailing, so I will introduce myself first. I'm François (alias Linkid), a French engineering student who code mainly in Python. I'm involved in some open source projects and some of them like Django or OpenStack.
I would like to add a test for Python 3 to check an import, but I don't really know on which project (pep8, pyflakes, …) to do that and how to do it (by a push request on github or somewhere else, by a bug report, …). I searched some clue and I think my test should be used by pyflakes, but I'm not sure… So, here are my questions: - on which project should I add a Python 3 import test ? - how I should process ?
I think it could help be a lot if someone explain me what one project really check, if it is possible.
Thank you.
François Magimel (Linkid) Hello François,
I'm not quite sure anyone on this list can answer your question. You haven't given us any guidance about what the check is that you would like to add. Do you want to check for imports that are not used in Python 3? Are you looking for imports that would fail on Python 3 instead? Are you looking to ensure some style of imports? Perhaps you're looking to do what openstack/hacking already does (ensure only modules are imported)? There are too many possibilities for us to be helpful.
Further, unless what you wish to add is part of http://legacy.python.org/dev/peps/pep-0008/, I can at least tell you that it does *not* belong in pep8. If you can give us more details I can also tell you whether it belongs in PyFlakes. If it turns out it does not belong in PyFlakes, you can always write and release your own plugin for Flake8.
Sorry I cannot be more helpful, Ian Hello,
Oh, yes, it will be better if I give some details. My code didn't pass my python 3 tests due to an import of string. As a matter of fact, I was using string.letters which is not in python 3 anymore. However, when I tested my code with flake8, I wasn't warned about it. So, I would like to add this warning (maybe to PyFlakes, but I'm not sure). And I think it is not only an OpenStack issue because it concerns every projects. So, if someone can help me to add it (to PyFlakes?) by giving me the process (PR on GitHub?), I will be very happy to contribute :).
Thank you for your answer, Ian. François
Can you please reply with the output from: flake8 --version On Mon, Aug 25, 2014 at 10:33 AM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
I tested with python 3.4, but Flake8 didn't recognized this error… But I thought Flake8 only runs pep8, PyFlakes & co. tests, no? So, if I would like to add this test, it may belong to PyFlakes, right?
Le 25/08/2014 14:38, Ian Cordasco a écrit :
So Flake8 would actually recognize that if you installed Flake8 under Python 3 (e.g., pip3 install flake8), and then run flake8 from Python 3.
You can do this with a virtualenv more easily like:
virtualenv -ppython3 path/to/virtualenv source path/to/virtualenv/bin/activate pip install flake8 flake8 project/
PyFlakes will only run checks for the version of Python it is installed on. Let me know if you run into any problems with this.
On Mon, Aug 25, 2014 at 3:36 AM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Le 25/08/2014 02:24, Ian Cordasco a écrit :
On Sun, Aug 24, 2014 at 3:07 PM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Hello!
I'm new in this mailing, so I will introduce myself first. I'm François (alias Linkid), a French engineering student who code mainly in Python. I'm involved in some open source projects and some of them like Django or OpenStack.
I would like to add a test for Python 3 to check an import, but I don't really know on which project (pep8, pyflakes, …) to do that and how to do it (by a push request on github or somewhere else, by a bug report, …). I searched some clue and I think my test should be used by pyflakes, but I'm not sure… So, here are my questions: - on which project should I add a Python 3 import test ? - how I should process ?
I think it could help be a lot if someone explain me what one project really check, if it is possible.
Thank you.
François Magimel (Linkid) Hello François,
I'm not quite sure anyone on this list can answer your question. You haven't given us any guidance about what the check is that you would like to add. Do you want to check for imports that are not used in Python 3? Are you looking for imports that would fail on Python 3 instead? Are you looking to ensure some style of imports? Perhaps you're looking to do what openstack/hacking already does (ensure only modules are imported)? There are too many possibilities for us to be helpful.
Further, unless what you wish to add is part of http://legacy.python.org/dev/peps/pep-0008/, I can at least tell you that it does *not* belong in pep8. If you can give us more details I can also tell you whether it belongs in PyFlakes. If it turns out it does not belong in PyFlakes, you can always write and release your own plugin for Flake8.
Sorry I cannot be more helpful, Ian Hello,
Oh, yes, it will be better if I give some details. My code didn't pass my python 3 tests due to an import of string. As a matter of fact, I was using string.letters which is not in python 3 anymore. However, when I tested my code with flake8, I wasn't warned about it. So, I would like to add this warning (maybe to PyFlakes, but I'm not sure). And I think it is not only an OpenStack issue because it concerns every projects. So, if someone can help me to add it (to PyFlakes?) by giving me the process (PR on GitHub?), I will be very happy to contribute :).
Thank you for your answer, Ian. François
Of course! Here is the return: $ flake8 --version 2.2.3 (pep8: 1.5.7, pyflakes: 0.8.1, mccabe: 0.2.1) CPython 3.4.1 on Linux And the code to test: import string print(string.letters) Le 25/08/2014 18:11, Ian Cordasco a écrit :
Can you please reply with the output from:
flake8 --version
On Mon, Aug 25, 2014 at 10:33 AM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
I tested with python 3.4, but Flake8 didn't recognized this error… But I thought Flake8 only runs pep8, PyFlakes & co. tests, no? So, if I would like to add this test, it may belong to PyFlakes, right?
Le 25/08/2014 14:38, Ian Cordasco a écrit :
So Flake8 would actually recognize that if you installed Flake8 under Python 3 (e.g., pip3 install flake8), and then run flake8 from Python 3.
You can do this with a virtualenv more easily like:
virtualenv -ppython3 path/to/virtualenv source path/to/virtualenv/bin/activate pip install flake8 flake8 project/
PyFlakes will only run checks for the version of Python it is installed on. Let me know if you run into any problems with this.
On Mon, Aug 25, 2014 at 3:36 AM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Le 25/08/2014 02:24, Ian Cordasco a écrit :
On Sun, Aug 24, 2014 at 3:07 PM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Hello!
I'm new in this mailing, so I will introduce myself first. I'm François (alias Linkid), a French engineering student who code mainly in Python. I'm involved in some open source projects and some of them like Django or OpenStack.
I would like to add a test for Python 3 to check an import, but I don't really know on which project (pep8, pyflakes, …) to do that and how to do it (by a push request on github or somewhere else, by a bug report, …). I searched some clue and I think my test should be used by pyflakes, but I'm not sure… So, here are my questions: - on which project should I add a Python 3 import test ? - how I should process ?
I think it could help be a lot if someone explain me what one project really check, if it is possible.
Thank you.
François Magimel (Linkid) Hello François,
I'm not quite sure anyone on this list can answer your question. You haven't given us any guidance about what the check is that you would like to add. Do you want to check for imports that are not used in Python 3? Are you looking for imports that would fail on Python 3 instead? Are you looking to ensure some style of imports? Perhaps you're looking to do what openstack/hacking already does (ensure only modules are imported)? There are too many possibilities for us to be helpful.
Further, unless what you wish to add is part of http://legacy.python.org/dev/peps/pep-0008/, I can at least tell you that it does *not* belong in pep8. If you can give us more details I can also tell you whether it belongs in PyFlakes. If it turns out it does not belong in PyFlakes, you can always write and release your own plugin for Flake8.
Sorry I cannot be more helpful, Ian Hello,
Oh, yes, it will be better if I give some details. My code didn't pass my python 3 tests due to an import of string. As a matter of fact, I was using string.letters which is not in python 3 anymore. However, when I tested my code with flake8, I wasn't warned about it. So, I would like to add this warning (maybe to PyFlakes, but I'm not sure). And I think it is not only an OpenStack issue because it concerns every projects. So, if someone can help me to add it (to PyFlakes?) by giving me the process (PR on GitHub?), I will be very happy to contribute :).
Thank you for your answer, Ian. François
On Wed, Aug 27, 2014 at 3:15 AM, François Magimel <francois.magimel@etu.enseeiht.fr> wrote:
Of course! Here is the return: $ flake8 --version 2.2.3 (pep8: 1.5.7, pyflakes: 0.8.1, mccabe: 0.2.1) CPython 3.4.1 on Linux
And the code to test:
import string
print(string.letters)
Oh, I see the confusion. So this passes because pyflakes will compile the code but that doesn't do attribute checking (that's done at runtime by Python), so the import works just fine. Pyflakes doesn't make assumptions about the attribute (letters in this case) and to add attribute checking to every module imported would significantly increase the complexity of the project. You would have to do several things: 1. Find where the module is being imported from (I can not remember if the AST holds this information, but I strongly doubt it does) 2. Load/compile the module in question 3. Attempt to statically determine if the attribute is available which may be in fact very error-prone. The first point sounds very simple, but is in fact not as simple as it might seem. More importantly to me, I'd like to focus on the third point. While most modules do not dynamically handle attributes there are plenty which do. One prime example of this is flask. For any extension of the framework, you can use entry points so that a user can do: from flask.ext import sqlalchemy Naturally the flask.ext module does not have every extension statically defined, in fact it has some very clever machinery to provide this interface to the user that would be far too complicated to detect or handle. Adding this kind of checking would ultimately be problematic and at times flat-out wrong. If you're still up for adding this, I'd strongly encourage you to try your hand adding it to PyFlakes. I'll even give you a lot of credit if you can get it to play nicely with modules like flask.ext.
Hi, first post here as well. I started lurking here a while ago because I am interested in static code analysis (atm strictly from a consumer perspecive though). On 28 August 2014 01:16, Ian Cordasco <graffatcolmingov@gmail.com> wrote:
$ flake8 --version 2.2.3 (pep8: 1.5.7, pyflakes: 0.8.1, mccabe: 0.2.1) CPython 3.4.1 on Linux
And the code to test:
import string
print(string.letters)
I'd like to add some pointers to other projects that might be better at solving problems like this. 1. I use PyCharm and it it has the best static code analysis I came across yet. It highlights errors like this ans is very clever with dynamically loaded imports as well (by collecting and caching runtime data as well). They have an open source community edition as well: http://www.jetbrains.com/pycharm/features/editions_comparison_matrix.html 1. Another interesting project is Jedi: https://github.com/davidhalter/jedi that started of as autocompletion lib, but is now moving towards static code analysis as well (and Dave is a nice guy as well :)). Cheers Oliver
participants (3)
-
François Magimel -
Ian Cordasco -
Oliver Bestwalter