Mailman 3 March 2020 - Distutils-SIG

Improving Communication
by Donald Stufft July 30, 2020

July 30, 2020

Currently in the packaging space, we have a number of avenues for communication, which are: - distutils-sig - pypa-dev - virtualenv-users - Other project specific mailing lists - IRC - gitter - Various issue trackers spread across multiple platforms. - Probably more places I’m not remembering. The result of this is that all discussion ends up being super fractured amongst the various places. Sometimes that is exactly what you want (for instance, someone who is working on the wheel specs probably doesn’t care about deprecation policy and internal module renaming in pip) and sometimes that ends up being the opposite of what you want (for instance, when you’re describing something that touches PyPI, setuptools, flit, pip, etc all at once). Theoretically the idea is that distutils-sig is where cross project reaching stuff goes, IRC/gitter is where real time discussion goes, and the various project mailing lists and issue trackers are where the project specific bits go. The problem is that often times doesn’t actually happen in practice except for the largest and most obvious of changes. I think our current “communications stack” kind of sucks, and I’d love to figure out a better way for us to handle this that solves the sort of weird “independent but related” set of projects we have here. From my POV, a list of our major problems are: * Discussion gets fractured across a variety of platforms and locations, which can make it difficult to actually keep up with what’s going on but also to know how to loop in someone relevant if their input would be valuable. You have to more or less simultaneously know someone’s email, Github username, IRC nick, bitbucket username, etc to be able to bring threads of discussion to people’s attention. * It’s not always clear to users where a discussion should go, often times they’ll come to one location and need to get redirected to another location. If any discussion did happen in the incorrect location, it tends to need to get restarted in the new location (and depending on the specific platform, it may be impossible to actually redirect everyone over to the proper location, so you again, end up fractured with the discussion happening in two places). * A lot of the technology in this stack is particularly old, and lacks a lot of the modern day affordances that newer things have. An example is being able to edit a discussion post to fix typos that can hinder the ability of others to actually understand whats being talked about. In your typical mailing list or IRC there’s no mechanism by which you can edit an already sent message, so your only option is to either let the problem ride and hope it doesn’t trip up too many people, or send an additional message to correct the error. However these show up as additional, later messages which someone might not even see until they’ve already been thoroughly confused by the first message (since people tend to read email/IRC in a linear fashion). - There is a lot of things in this one, other things are things like being able to control in a more fine grained manner what email you’re going to get. - Holy crap, formatting and typography to make things actually readable and not a big block of plaintext. * We don’t have a clear way for users to get help, leaving users to treat random issues, discussion areas, etc as a support forum, rather than some place that’s actually optimized for that. Some of this ties back into some of the earlier things too, where it’s hard to actually redirect discussions These aren’t *new* problems, and often times the existing members of a community are the least effected becasue they’ve already spent effort learning the ins and outs and also curating a (typically custom) workflow that they’ve grown accustomed too. The problem with that is that often times that means that new users are left out, and the community gets smaller and smaller as time goes on as people leave and aren’t replaced with new blood, because they’re driven off but the issues with the stack. A big part of the place this is coming from, is me sitting back and realizing that I tend to be drawn towards pulling discussions into Github issues rather than onto the varying mailing lists, not because that’s always the most appropriate place for it, but because it’s the least painful place in terms of features and functionality. I figure if I’m doing that, when I already have a significant investment in setting up tooling and being involved here, that others (and particularly new users) are likely feeling the same way. - Donald

9 12

FASTEN and dependency analysis at call graph level
by Sumana Harihareswara July 16, 2020

July 16, 2020

Last week I heard for the first time about the research project FASTEN https://www.fasten-project.eu/. "FASTEN stands for Fine-Grained Analysis of Software Ecosystems as Networks." > instead of analyzing dependencies at the package level, we will analyze them at the call graph level! This will allow us to be super precise when we are tracking dependencies, when we do change impact analysis, when we recommend clients to update packages etc. It will also open the door to new sophisticated applications, e.g. licensing compliance, dependency risk profiling and data-driven API evolution. That's from the blog post by Georgios Gousios, the PI, at http://www.gousios.gr/blog/Introducing-Fasten.html . More info: https://www.fasten-project.eu/view/Main/Introduction https://www.fasten-project.eu/view/Main/Overview https://www.fasten-project.eu/view/Main/Contacts And people who are interested in dynamically and statically analyzing call graphs in Python may be interested in "Graph Schema and its representation" in https://www.fasten-project.eu/view/Main/Deliverables . I've sent a note to FASTEN inviting the team to come talk about their project here on distutils-sig, because FASTEN's site says they aim to eventually integrate into PyPI -- I'm not 100% sure whether that means "create a service that people can use WITH PyPI" or "get FASTEN's work incorporated into pypi.org". -- Sumana Harihareswara pip project manager on contract with Python Software Foundation Changeset Consulting https://changeset.nyc

3 2

PyPi not allowing duplicate filenames
by Chris Barker April 20, 2020

April 20, 2020

Hi all, PyPi does not allow duplicate file names -- this makes lots of sense, because you really don't want people to go to PyPi one day and grab a file, and then go there another day, grab a file with exactly the same name, and have it be a different file. However.... We are all too human, and make mistakes when doing a release. All to often someone pushed a broken file up to PyPi, often realizes it pretty quickly -- before anyone has a chance to even download it (or only the dev team as, for testing...). In fact, I was in a sprint last summer, and we decided to push our package up to PyPi -- granted, we were all careless amateurish noobs, but we ended up making I think 4! minor version bumps because we had done something stupid in the sdist. Also -- the latest numpy release got caught in this, too: """ * We ran into a problem with pipy not allowing reuse of filenames and a resulting proliferation of *.*.*.postN releases. Not only were the names getting out of hand, some packages were unable to work with the postN suffix. """ So -- I propose that PyPi allow projects to replace existing files if they REALLY REALLY want to. You should have to jump through all sorts of hoops, and make it really clear that it is a BAD IDEA in the general case, but it'd be good to have it be possible. After all -- PyPi does not take on responsibility for anything else about what's in those packages, and Python itself is all about "we're all consenting adults here" I suppose we could even put in some heuristics about how long the file as been there, how many times it's been downloaded, etc. Just a thought.....I really hate systems that don't let me roll back mistakes, even when I discover them almost immediately... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

13 24

ImportError running 'test' on namespace package - other package in namespace not found
by Jason R. Coombs April 17, 2020

April 17, 2020

I've got two projects: mynamespace.myprojectA and mynamespace.myprojectB myprojectB depends on myprojectA. I'm using setuptools 0.6c8 to manage both projects. Both projects are registered using 'setup develop'. Both projects are accessible from an interactive interpreter: PS C:\Users\me\projects> python Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import mynamespace.myprojectA >>> import mynamespace.myprojectB >>> from mynamespace.myprojectA import mymoduleZ However, when I run 'setup test' in myprojectB, the tests fail with File ".mymoduleZ.py", line NNN, in [some context] from mynamespace.myprojectA.mymoduleZ import MyClassC ImportError: No module named myprojectA.mymoduleZ In setup.py, the test_suite is nose.collector. I searched and couldn't find anyone else with this problem. Is this a supported configuration? Is there something I can do to make tests work with interdependent projects with the same root namespace? If there's not something obvious I should be doing differently, I'm happy to put together a minimal test case that reproduces the problem. Any suggestions are appreciated. Sincerely, Jason R. Coombs

3 6

Preview: Upcoming changes to PyPI BigQuery public dataset
by Ernest W. Durbin III March 30, 2020

March 30, 2020

Hello! I’m looking for anyone how has built something interesting using the PyPI Big Query public dataset that’s documented at packaging.python.org/guides/analyzing-pypi-package-downloads. There are some changes that are coming up that we’d like to preview for users that are actively using the dataset. If that’s you, send a note to bigquery-feedback(a)pypi.org with how you’ve been using the dataset. I’ll get back to you with information on the upcoming changes and will select a few projects to feature in the announcement of the new changes. -Ernest W. Durbin III Director of Infrastructure Python Software Foundation

1 0

Install distutils with pip3
by Samuel Mutel March 28, 2020

March 28, 2020

Hello, I would like to know if distutils could be installed in python3 environment with pip3 ? Thanks.

2 1

Pipenv - Use system python lib for one package
by Samuel Mutel March 28, 2020

March 28, 2020

Hello, I have a pipfile like this. I would like to install python-apt library but not from pypi but from the system. Is-it possible? This python library is no longer supported on pypi but is supported in the package distribution. [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev-packages] [packages] ansible = "==2.8.4" ansible-lint = "*" molecule = "*" docker-py = "*" [requires] python_version = "3"

2 4

Next Pipenv Release
by Dan Ryan March 27, 2020

March 27, 2020

Hey all -- With Sumana's help, I've been working to get the next pipenv release out by the end of the month. As some of you may already know, I recently changed jobs and am now working for Canonical where I have part of my time allocated to open source work in general (and pipenv in particular). The main pipenv release blockers have been fixes in andnew releases of requirementslib, pythonfinder, pip-shims, and vistir, and merging https://github.com/pypa/pipenv/tree/feature/vendor-updateto address missing dependencies/imports.I've beenworking on these and should wrap up this week.|[0]| You can help in a few ways: 1. Look at the tracking issue for the new release https://github.com/pypa/pipenv/issues/3369and speak up there if you know of a release blocker that isn't in the list. 2. If you see me in #pypa-dev on Freenode IRC, feel free to ask me what I'm working on. Sumana's been keeping me focused this month (and helped me write up the tracking issue, and this email) and it's been helpful, as I am juggling a lot of obligations and can get distracted.|[1]| 3. I could use help replying to new users' questions in pipenv GitHub issues, so I can concentrate on the new release. If you'd like triage permissions on the project please reach out, I'd be happy to give them. 4. Documentation! Pipenv documentation, now at https://pipenv.pypa.io/,needs some serious rework. So if you have any skills in this area,the project would really benefit from a critical review here. 5. Make sure to say 'thanks' to Sumana if you see her on IRC, she is responsible for moving this release forward and is pretty great! I'd appreciate the help. Thanks, -Dan Ryan (techalchemy) |[0]:| You can look at https://github.com/pypa/pipenv/issues/3742#issuecomment-492100711and https://github.com/pypa/pipenv/issues/4058#issuecomment-565550646for more context on why we've been delayed on a new release since November 2018. |[1]|: Iacknowledge that I've been super behind on this release,but I also have moved, had a fire in my new house, changed jobs, etc., and am onlynowsettling into my new role where I finally am paid to spend some time on pipenv.So, after this release is out, I hope to get back to helping new contributors more so we can promote some more co-maintainers, so this doesn't block releases again. -- Dan Ryan @techalchemy | Pipenv Maintainer

2 1

(no subject)
by aaa rrr March 26, 2020

March 26, 2020

مؤسسته

1 0

PEP 440
by Jose Quevedo March 25, 2020

March 25, 2020

1 0

comment avoir Matlibplot sur Python 3.8.2 sur MAC OS Mojave ?
by moise amsellem March 25, 2020

March 25, 2020

1 0

PEP 440
by Rosemary Gutierrez March 25, 2020

March 25, 2020

1 0

pip resolver work chugging along
by Sumana Harihareswara March 24, 2020

March 24, 2020

The alpha or beta release of pip with its new dependency resolver should be out in May. I just posted https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year… which discusses what is going to change in the pip resolver, when, and how you can help (including some low-effort things you can do right now). I didn't mention this in the blog post because ordinary Python users shouldn't try it, but: As of right now, people who install pip from GitHub master will have the ability to run `pip install --unstable-feature=resolver` and test the new resolver code. And less than half of the test suite fails! Expect errors and missing features, but it’s there! [Celebratory trumpet honk here.] Hope all of you, and all the people you are close to, are healthy and staying that way. -- Sumana Harihareswara pip project manager (contracting with Python Software Foundation) Changeset Consulting https://changeset.nyc

5 11

(no subject)
by 郭罗华 March 24, 2020

March 24, 2020

发送自 Windows 10 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>应用

1 0

(no subject)
by وائل عبدالمولى March 23, 2020

March 23, 2020

1 0

Questionable package in PyPi, maybe malicious
by Evagelos March 21, 2020

March 21, 2020

Hi there, I searched PyPi for "flaskmigrate" and this came up: flaskmigrate - A package to prevent exploit Seems strange to me so I wanted to make you aware, I was advised to report in this mailing list. Regards, Evagelos

2 3

My email address
by Evagelos Theodoridis March 21, 2020

March 21, 2020

evagelos.theo(a)gmail.com

2 1

Today: livestreamed talk about PyPI malware detection
by Sumana Harihareswara March 16, 2020

March 16, 2020

Today at 1pm PT/4pm ET: a livestreamed presentation by Cristina Muñoz, who's been working on the PyPI malware detection feature: https://www.meetup.com/pacifichackers/events/267932809/ "Automatic Detection of Malware in PyPI" Alternate link: https://phack.my.webex.com/phack.my/j.php?MTID=mdb827dc0a7f6dfe9784f793686e… She noted: > A general note: this is a presentation geared more towards security folks. A lot of the Python stuff I talk about might feel really redundant/obvious for people who are software engineers and have Python familiarity. Like, there are several slides describing what PyPI is, and the difference between packages, releases and files, for example. -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

1 1

replacement for tests_require?
by Felix Schwarz March 9, 2020

March 9, 2020

Hi, what is the recommended to solution to declare dependencies which are only required to run tests? In the past there was "tests_require" but as far as I know this is deprecated. I tried to read obvious github issues but maybe I missed something so please bear with me if this has come up way too often. I appreciate pointers. Felix

3 3

Provide a way to bundle and extract license files
by Steve Jorgensen March 7, 2020

March 7, 2020

The project I am working on now needs to include license files for all of the 3rd party code that it includes. Since license files are not included in distribution packages, the process for doing this is exceedingly complex, error prone, and in some cases, impossible. The build process builds URLs to license files in package source code repositories based on version number, but this is dependent upon several things… 1. The source code repository is accessible via the Internet and still exists. 2. There is a tag for each version (not always true). 3. Tags follow a consistent naming pattern with respect to version numbers (not always true). I believe that it makes sense to provide a standard means for distribution packages to include license files and then encourage package authors to use it.

4 8

[setuptools] Install entry point only if extras_require are satisfied?
by Jonatan Palsson March 6, 2020

March 6, 2020

Hi, I'm modifying a python project which uses setuptools for installation. The project installs an entry point as such: setup_args["entry_points"] = { "console_scripts": ["w1thermsensor = w1thermsensor.cli:cli [CLI]"] } where [CLI] is a reference to an extra_requires entry. This entry looks like this: extras_require=dict(CLI="click>=7.0") With these two lines, the current behavior is that the w1thermsensor entry point is *always* installed when "setup.py install" is invoked, but the entry point will cause an error if its dependencies are not available. I would like to change the behavior, so that the entry point is *not installed* if the extra dependencies are not available (and perhaps also show a warning, indicating that this entry point has not been installed). How can I do this? Cheers, Jonatan

2 1

The base docker images for manylinux appear to be private
by Sylvain Corlay March 1, 2020

March 1, 2020

It seems that the base images used for manylinux wheels are not publicly available. For example, https://quay.io/pypa/manylinux2010_centos-6-no-vsyscall gives a 403 error ("you are not authorized to view this resource"). I think that it would make sense for reproducibility purposes to make these images public. (I have opened an issue about this on the GitHub repository: https://github.com/pypa/manylinux/issues/481). Best,

5 8