Hi Ingrid,

I think this could be an extremely valuable tool for anyone in need of maintaining a Python application.

IMHO, being able to identify risks of upgrade errors based on type signatures of the public API of the dependencies we are using would be a game changer.

Security checks would be also useful, but I believe there are already tools that can help with that.

Regarding integration with pip: this would be nice, but I don't believe this is mandatory. The current generation of tools that help upgrade and pin dependencies (pip-tools, Poetry...) provide their own CLI, instead of extending pip. So both options are possible.



On Thu, Jul 16, 2020 at 3:37 PM Ingrid Sena <job@ingridsena.com.br> wrote:
Hi Sumana,
Thank you for reaching out to the project team.

We do want to be integrated to package managers such as pip, and currently, we are in the early stages to develop the integration plan. It would be great to have feedback and collaboration with the python community on how our approach could be since the goal of the project is linked to several distinct ongoing discussions and PEPs.

Please find more details below:

Project overall concept:
FASTEN stands for Fine-Grained Analysis of Software Ecosystems as Networks.
We aim to make software ecosystems more robust by making package management more intelligent. The core idea that FASTEN relies upon is the creation of an ecosystem-wide Fine-Grained Call Graph (FGCG), at the function level. FASTEN will perform sophisticated analyses of i) security vulnerability propagation, ii) licensing compliance, and iii) dependency risk profiles. The result is a versioned, ecosystem-level call graph, that not only solves the issues identified above but also both opens the doors to advanced applications and challenges the current state of the art in graph storage and processing. To facilitate adoption, FASTEN will bring those analyses to the hands of developers by integrating the analysis service to popular package managers, for the Java, C, and Python programming languages.

More info here: https://www.fasten-project.eu/view/Main/Overview

pip integration proposal:
A central aspect of the project is the promise to bring the power of the analyses afforded by the fine-grained ecosystem approach to the hands of the developers. Moreover, the package manager plug-ins will allow Continuous Integration servers and other components in Continuous Delivery pipelines to automate tasks such as compliance and security checks while a project is built (even behind corporate firewalls). For those reasons, we will create a plugin for pip and it will include the ability to produce call graphs compatible with FASTEN and instrument the host virtual machine to produce dynamic call graphs. The plug-in will interact with the FASTEN’s REST API. An example session with a FASTEN-enabled package manager can be seen below:

> pip list
docutils (0.10)
Jinja2 (2.7.2)
MarkupSafe (0.18)

> pip check-security
Jinja2 (2.7.2) has known vulnerabilities (your project is affected!)
Update to version >=2.7.3 (will not break your project)

> pip test-upgrade Jinja2 --version 2.8
Upgrading to Jinja2 2.8 will break the following methods:

> pip what-breaks --delete myproject.foo
The following direct dependencies will break if you *delete* function foo()
* projectA: 15 methods use foo()
* projectB: 10 methods use foo()
632 indirect dependencies will fail to work.

> pip test --upload-dyngraph
............15 Tests run OK!
Dynamic call graph at: myproject.dot
Uploading dynamic call graph to FASTEN

There is currently no such knowledge base in existence, let alone one with the level of detail that FASTEN aims for. Through integrations with existing services (e.g. GitHub) and developer tools (e.g. mvn and pip), the FASTEN Knowledge Base will make OSS software ecosystems better as a whole by increasing developer awareness and minimizing manual work that developers have to do to keep their projects up to date. The FASTEN consortium foresees that the Knowledge Base and its open REST API, will lead to a flourishing ecosystem of tools and services that take advantage of it; for this reason, FASTEN will work to create a community of contributing users around it.

Our github: https://github.com/fasten-project/fasten
FASTEN Overview presentation at OW2online20 conference: https://bittube.video/videos/watch/e16ff292-cfbe-4b44-86e2-0c1b040d5f9c

Ingrid Sena
Software Engineer @ Endocode AG

Endocode AG, Brückenstraße 5A, 10179 Berlin
+49 30 1206 4472 | info@endocode.com | www.endocode.com

Vorstand: Lisa Nöth (Vorsitzende), Sebastian Sucker
Aufsichtsratsvorsitzender: Mirko Boehm

Registergericht: Amtsgericht Charlottenburg - HRB 150748 B
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-leave@python.org
Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/ZCV7OTM5WADBFE7XV6NRQC46PWCRHYYP/

Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/
Chairman, National Council for Free & Open Source Software (CNLL) - http://cnll.fr/
Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/http://pydata.fr/