Hello everyone! This is my first post to this list, so first, let me give a quick introduction. I'm VP of Engineering at GrammaTech, where I am in charge of an advanced static analysis tool named CodeSonar. It primarily works for C and C++, but also for x86, x64 and ARM binaries. We cover other languages by integrating with other tools (mostly open source). We don't have an integration with Pylint yet, but that's coming as described below. I'm writing to let the community know of some work we will be doing that should benefit everyone. I think I know the best way forward, but I'd appreciate any words of wisdom and feedback on our approach. SARIF stands for Static Analysis Results Interchange Format. It is a new standard that originated at Microsoft, and that is now under the OASIS umbrella (I'm on the TC): https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sarif. The idea is to make it easier for tools that produce results to integrate with tools that consume results. Our own tool is both a producer and a consumer. That is, it can import results from SARIF-compatible tools and show them it is user interface. Our strategy to make CodeSonar be useful for other languages is through SARIF; we'll write converters to SARIF for the best-of-breed tools. Consequently, we are planning to make it so that Pylint can produce SARIF. There are two good ways to do this. 1. The easiest thing to do is to simply run "pylint -f json ..." and write a simple program to convert the output to SARIF (data from "pylint --list-msgs" is also needed). We're doing this first. A nice thing about this approach is that it doesn't require any changes to Pylint. The disadvantage is that it's likely to be very sensitive to the particular version of Pylint used. E.g., if the format of those outputs change. The plan is to contribute this to the sarif SDK github. 2. The better long-term approach is to change pylint to add a new output format so one can do "pylint -f sarif ...". This way, everyone gets it. I'm not expecting this to be too difficult, although I concede that I haven't scrutinized the pylint code enough to know for sure. I'm expecting #1 to appear within a couple of weeks, and to start work on #2 by the end of the month. I'd appreciate any input from interested parties. Thanks, -Paul -- Paul Anderson, VP of Engineering, GrammaTech, Inc. 531 Esty St., Ithaca, NY 14850 Tel: +1 607 273-7340 x118; http://www.grammatech.com
Hi Paul, Nice to meet you. Thank you for that link, I wasn't aware of this new standard. It would be great to support it for pylint as well, and most likely shouldn't be too difficult to add a custom reporter in the same vein as the JSON reporter. You can find some examples here: https://github.com/PyCQA/pylint/tree/master/pylint/reporters Cheers, Claudiu On 3 September 2018 at 16:59, Paul Anderson <paul@grammatech.com> wrote:
Hello everyone!
This is my first post to this list, so first, let me give a quick introduction. I'm VP of Engineering at GrammaTech, where I am in charge of an advanced static analysis tool named CodeSonar. It primarily works for C and C++, but also for x86, x64 and ARM binaries. We cover other languages by integrating with other tools (mostly open source). We don't have an integration with Pylint yet, but that's coming as described below.
I'm writing to let the community know of some work we will be doing that should benefit everyone. I think I know the best way forward, but I'd appreciate any words of wisdom and feedback on our approach.
SARIF stands for Static Analysis Results Interchange Format. It is a new standard that originated at Microsoft, and that is now under the OASIS umbrella (I'm on the TC): https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sarif. The idea is to make it easier for tools that produce results to integrate with tools that consume results. Our own tool is both a producer and a consumer. That is, it can import results from SARIF-compatible tools and show them it is user interface. Our strategy to make CodeSonar be useful for other languages is through SARIF; we'll write converters to SARIF for the best-of-breed tools.
Consequently, we are planning to make it so that Pylint can produce SARIF. There are two good ways to do this.
1. The easiest thing to do is to simply run "pylint -f json ..." and write a simple program to convert the output to SARIF (data from "pylint --list-msgs" is also needed). We're doing this first. A nice thing about this approach is that it doesn't require any changes to Pylint. The disadvantage is that it's likely to be very sensitive to the particular version of Pylint used. E.g., if the format of those outputs change. The plan is to contribute this to the sarif SDK github.
2. The better long-term approach is to change pylint to add a new output format so one can do "pylint -f sarif ...". This way, everyone gets it. I'm not expecting this to be too difficult, although I concede that I haven't scrutinized the pylint code enough to know for sure.
I'm expecting #1 to appear within a couple of weeks, and to start work on #2 by the end of the month. I'd appreciate any input from interested parties.
Thanks,
-Paul
-- Paul Anderson, VP of Engineering, GrammaTech, Inc. 531 Esty St., Ithaca, NY 14850 Tel: +1 607 273-7340 x118; http://www.grammatech.com
_______________________________________________ code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality
I think Bandit (https://github.com/pycqa/bandit) is another project we should add this to, as well as Flake8 On Mon, Sep 3, 2018 at 10:25 AM Claudiu Popa <pcmanticore@gmail.com> wrote:
Hi Paul,
Nice to meet you. Thank you for that link, I wasn't aware of this new standard. It would be great to support it for pylint as well, and most likely shouldn't be too difficult to add a custom reporter in the same vein as the JSON reporter. You can find some examples here: https://github.com/PyCQA/pylint/tree/master/pylint/reporters
Cheers, Claudiu
On 3 September 2018 at 16:59, Paul Anderson <paul@grammatech.com> wrote:
Hello everyone!
This is my first post to this list, so first, let me give a quick introduction. I'm VP of Engineering at GrammaTech, where I am in charge of an advanced static analysis tool named CodeSonar. It primarily works for C and C++, but also for x86, x64 and ARM binaries. We cover other languages by integrating with other tools (mostly open source). We don't have an integration with Pylint yet, but that's coming as described below.
I'm writing to let the community know of some work we will be doing that should benefit everyone. I think I know the best way forward, but I'd appreciate any words of wisdom and feedback on our approach.
SARIF stands for Static Analysis Results Interchange Format. It is a new standard that originated at Microsoft, and that is now under the OASIS umbrella (I'm on the TC): https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sarif. The idea is to make it easier for tools that produce results to integrate with tools that consume results. Our own tool is both a producer and a consumer. That is, it can import results from SARIF-compatible tools and show them it is user interface. Our strategy to make CodeSonar be useful for other languages is through SARIF; we'll write converters to SARIF for the best-of-breed tools.
Consequently, we are planning to make it so that Pylint can produce SARIF. There are two good ways to do this.
1. The easiest thing to do is to simply run "pylint -f json ..." and write a simple program to convert the output to SARIF (data from "pylint --list-msgs" is also needed). We're doing this first. A nice thing about this approach is that it doesn't require any changes to Pylint. The disadvantage is that it's likely to be very sensitive to the particular version of Pylint used. E.g., if the format of those outputs change. The plan is to contribute this to the sarif SDK github.
2. The better long-term approach is to change pylint to add a new output format so one can do "pylint -f sarif ...". This way, everyone gets it. I'm not expecting this to be too difficult, although I concede that I haven't scrutinized the pylint code enough to know for sure.
I'm expecting #1 to appear within a couple of weeks, and to start work on #2 by the end of the month. I'd appreciate any input from interested parties.
Thanks,
-Paul
-- Paul Anderson, VP of Engineering, GrammaTech, Inc. 531 Esty St., Ithaca, NY 14850 Tel: +1 607 273-7340 x118; http://www.grammatech.com
_______________________________________________ code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality
code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality
Ian: I'll take a look at those too, but I can't promise I'll have the bandwidth for them. -Paul On 9/3/2018 12:47 PM, Ian Stapleton Cordasco wrote:
I think Bandit (https://github.com/pycqa/bandit) is another project we should add this to, as well as Flake8 On Mon, Sep 3, 2018 at 10:25 AM Claudiu Popa <pcmanticore@gmail.com> wrote:
Hi Paul,
Nice to meet you. Thank you for that link, I wasn't aware of this new standard. It would be great to support it for pylint as well, and most likely shouldn't be too difficult to add a custom reporter in the same vein as the JSON reporter. You can find some examples here: https://github.com/PyCQA/pylint/tree/master/pylint/reporters
Cheers, Claudiu
On 3 September 2018 at 16:59, Paul Anderson <paul@grammatech.com> wrote:
Hello everyone!
This is my first post to this list, so first, let me give a quick introduction. I'm VP of Engineering at GrammaTech, where I am in charge of an advanced static analysis tool named CodeSonar. It primarily works for C and C++, but also for x86, x64 and ARM binaries. We cover other languages by integrating with other tools (mostly open source). We don't have an integration with Pylint yet, but that's coming as described below.
I'm writing to let the community know of some work we will be doing that should benefit everyone. I think I know the best way forward, but I'd appreciate any words of wisdom and feedback on our approach.
SARIF stands for Static Analysis Results Interchange Format. It is a new standard that originated at Microsoft, and that is now under the OASIS umbrella (I'm on the TC): https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sarif. The idea is to make it easier for tools that produce results to integrate with tools that consume results. Our own tool is both a producer and a consumer. That is, it can import results from SARIF-compatible tools and show them it is user interface. Our strategy to make CodeSonar be useful for other languages is through SARIF; we'll write converters to SARIF for the best-of-breed tools.
Consequently, we are planning to make it so that Pylint can produce SARIF. There are two good ways to do this.
1. The easiest thing to do is to simply run "pylint -f json ..." and write a simple program to convert the output to SARIF (data from "pylint --list-msgs" is also needed). We're doing this first. A nice thing about this approach is that it doesn't require any changes to Pylint. The disadvantage is that it's likely to be very sensitive to the particular version of Pylint used. E.g., if the format of those outputs change. The plan is to contribute this to the sarif SDK github.
2. The better long-term approach is to change pylint to add a new output format so one can do "pylint -f sarif ...". This way, everyone gets it. I'm not expecting this to be too difficult, although I concede that I haven't scrutinized the pylint code enough to know for sure.
I'm expecting #1 to appear within a couple of weeks, and to start work on #2 by the end of the month. I'd appreciate any input from interested parties.
Thanks,
-Paul
-- Paul Anderson, VP of Engineering, GrammaTech, Inc. 531 Esty St., Ithaca, NY 14850 Tel: +1 607 273-7340 x118; http://www.grammatech.com
_______________________________________________ code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality
code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality
-- Paul Anderson, VP of Engineering, GrammaTech, Inc. 531 Esty St., Ithaca, NY 14850 Tel: +1 607 273-7340 x118; http://www.grammatech.com
Claudiu: On 9/3/2018 11:25 AM, Claudiu Popa wrote:
Hi Paul,
Nice to meet you. Thank you for that link, I wasn't aware of this new standard. It would be great to support it for pylint as well, and most likely shouldn't be too difficult to add a custom reporter in the same vein as the JSON reporter. You can find some examples here: https://github.com/PyCQA/pylint/tree/master/pylint/reporters I'll take a look there and let you know if I have any questions.
-Paul
Cheers, Claudiu
On 3 September 2018 at 16:59, Paul Anderson <paul@grammatech.com> wrote:
Hello everyone!
This is my first post to this list, so first, let me give a quick introduction. I'm VP of Engineering at GrammaTech, where I am in charge of an advanced static analysis tool named CodeSonar. It primarily works for C and C++, but also for x86, x64 and ARM binaries. We cover other languages by integrating with other tools (mostly open source). We don't have an integration with Pylint yet, but that's coming as described below.
I'm writing to let the community know of some work we will be doing that should benefit everyone. I think I know the best way forward, but I'd appreciate any words of wisdom and feedback on our approach.
SARIF stands for Static Analysis Results Interchange Format. It is a new standard that originated at Microsoft, and that is now under the OASIS umbrella (I'm on the TC): https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=sarif. The idea is to make it easier for tools that produce results to integrate with tools that consume results. Our own tool is both a producer and a consumer. That is, it can import results from SARIF-compatible tools and show them it is user interface. Our strategy to make CodeSonar be useful for other languages is through SARIF; we'll write converters to SARIF for the best-of-breed tools.
Consequently, we are planning to make it so that Pylint can produce SARIF. There are two good ways to do this.
1. The easiest thing to do is to simply run "pylint -f json ..." and write a simple program to convert the output to SARIF (data from "pylint --list-msgs" is also needed). We're doing this first. A nice thing about this approach is that it doesn't require any changes to Pylint. The disadvantage is that it's likely to be very sensitive to the particular version of Pylint used. E.g., if the format of those outputs change. The plan is to contribute this to the sarif SDK github.
2. The better long-term approach is to change pylint to add a new output format so one can do "pylint -f sarif ...". This way, everyone gets it. I'm not expecting this to be too difficult, although I concede that I haven't scrutinized the pylint code enough to know for sure.
I'm expecting #1 to appear within a couple of weeks, and to start work on #2 by the end of the month. I'd appreciate any input from interested parties.
Thanks,
-Paul
-- Paul Anderson, VP of Engineering, GrammaTech, Inc. 531 Esty St., Ithaca, NY 14850 Tel: +1 607 273-7340 x118; http://www.grammatech.com
_______________________________________________ code-quality mailing list code-quality@python.org https://mail.python.org/mailman/listinfo/code-quality
-- Paul Anderson, VP of Engineering, GrammaTech, Inc. 531 Esty St., Ithaca, NY 14850 Tel: +1 607 273-7340 x118; http://www.grammatech.com
participants (3)
-
Claudiu Popa -
Ian Stapleton Cordasco -
Paul Anderson