Mailman 3 October 2015 - Distutils-SIG

(no subject)
by Maciej Szulik 26 Oct '15

26 Oct '15

Thanks to Nick Coghlan and Barry Warsaw we've setup a new SIG dedicated to discussing python features from different distributions point of view. Here is Nick's reasoning: > With the Python 3 migration, and the growth in interest in user level > package management for development purposes, what do you think of the idea > of setting up a new Linux SIG to have those discussions? I know it's a case > of "yet another mailing list", but I think it will be worthwhile to have a > clear point of collaboration within the Python ecosystem, rather than > expecting Pythonistas to know how to reach out to (other) distros directly. The list is available @ https://mail.python.org/mailman/listinfo/linux-sig Maciej

1 0

Re: [Distutils] Remove distutils, was: red, green, refactor ...
by Thomas Güttler 26 Oct '15

26 Oct '15

Am 22.10.2015 um 18:34 schrieb Chris Barker: > On Thu, Oct 22, 2015 at 8:46 AM, Thomas Güttler <guettliml(a)thomas-guettler.de <mailto:guettliml@thomas-guettler.de>> wrote: > > I prefer the code of unittests to the not testable sentences in a standard. > > > sure -- but unittests are essentially an implementation of a standard -- you need a standard to write the unit tests for. You need a standard to write the unit test for .... If I look at the all the unittests in python itself. What is your guess: How many (percentage) exist to ensure the compatibility with a standard, and how many exist to ensure things work and bugs don't appear again? Unittest can be written to ensure a road map or "rough consensus and working code". Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/

1 0

Is this a bug?
by Nagy László Zsolt 26 Oct '15

26 Oct '15

Today I have tried to register and upload a new package by executing setup.py register I was asked if I want to save the creditentials in a .pypirc file and I answered yes. Next I wanted to run setup.py upload and I got this error: Traceback (most recent call last): File "C:\Python\Projects\some_package\setup.py", line 15, in <module> classifiers=['Topic :: Security', 'Topic :: Internet :: WWW/HTTP'], File "C:\Python35\lib\distutils\core.py", line 148, in setup dist.run_commands() File "C:\Python35\lib\distutils\dist.py", line 955, in run_commands self.run_command(cmd) File "C:\Python35\lib\distutils\dist.py", line 973, in run_command cmd_obj.ensure_finalized() File "C:\Python35\lib\distutils\cmd.py", line 107, in ensure_finalized self.finalize_options() File "C:\Python35\lib\distutils\command\upload.py", line 46, in finalize_options config = self._read_pypirc() File "C:\Python35\lib\distutils\config.py", line 83, in _read_pypirc current[key] = config.get(server, key) File "C:\Python35\lib\configparser.py", line 798, in get d) File "C:\Python35\lib\configparser.py", line 396, in before_get self._interpolate_some(parser, option, L, value, section, defaults, 1) File "C:\Python35\lib\configparser.py", line 445, in _interpolate_some "found: %r" % (rest,)) configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found : *****************' Instead of the many stars, of course there is the password. The problem might be that the password contains a % character, and it was incorrectly saved by distutils. Can somebody please confirm that this is a bug in distutils? Thanks, Laszlo

2 3

py2dsc --with-python3=True not producing python3- packages
by Marques Johansson 22 Oct '15

22 Oct '15

I assume I am doing something wrong, but the man page seems a bit light on the matter. Does it have something to do with "python setup.py" being called instead of "python3 setup.py". I have python3-stdeb=0.8.2-4 installed. wget https://pypi.python.org/packages/source/F/Flask-Login/Flask-Login-0.3.2.tar… py2dsc --with-python3=True --no-python2-scripts=True --no-python3-scripts=False --with-python2=False Flask-Login-0.3.2.tar.gz -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- running the following command in directory: deb_dist/tmp_py2dsc/Flask-Login-0.3.2 /usr/bin/python setup.py --command-packages stdeb.command sdist_dsc --dist-dir=/home/marques/src/deb_dist --use-premade-distfile=/home/marques/src/Flask-Login-0.3.2.tar.gz --with-python2=False --with-python3=True --no-python2-scripts=True --no-python3-scripts=False -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- running sdist_dsc running egg_info writing requirements to Flask_Login.egg-info/requires.txt writing Flask_Login.egg-info/PKG-INFO writing top-level names to Flask_Login.egg-info/top_level.txt writing dependency_links to Flask_Login.egg-info/dependency_links.txt reading manifest file 'Flask_Login.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'Flask_Login.egg-info/SOURCES.txt' CALLING dpkg-source -b flask-login-0.3.2 flask-login_0.3.2.orig.tar.gz (in dir /home/marques/src/deb_dist) dpkg-source: info: using source format `3.0 (quilt)' dpkg-source: info: building flask-login using existing ./flask-login_0.3.2.orig.tar.gz dpkg-source: info: building flask-login in flask-login_0.3.2-1.debian.tar.xz dpkg-source: info: building flask-login in flask-login_0.3.2-1.dsc dpkg-buildpackage: source package flask-login dpkg-buildpackage: source version 0.3.2-1 dpkg-buildpackage: source distribution unstable dpkg-buildpackage: source changed by Matthew Frazier < leafstormrush(a)gmail.com> dpkg-source --before-build flask-login-0.3.2 fakeroot debian/rules clean dh clean --with python3 --buildsystem=pybuild dh_testdir -O--buildsystem=pybuild dh_auto_clean -O--buildsystem=pybuild I: pybuild base:170: python3.4 setup.py clean running clean removing '/home/marques/src/deb_dist/flask-login-0.3.2/.pybuild/pythonX.Y_3.4/build' (and everything under it) 'build/bdist.linux-x86_64' does not exist -- can't clean it 'build/scripts-3.4' does not exist -- can't clean it dh_clean -O--buildsystem=pybuild dpkg-source -b flask-login-0.3.2 dpkg-source: info: using source format `3.0 (quilt)' dpkg-source: info: building flask-login using existing ./flask-login_0.3.2.orig.tar.gz dpkg-source: warning: ignoring deletion of directory Flask_Login.egg-info dpkg-source: warning: ignoring deletion of file Flask_Login.egg-info/dependency_links.txt, use --include-removal to override dpkg-source: warning: ignoring deletion of file Flask_Login.egg-info/not-zip-safe, use --include-removal to override dpkg-source: warning: ignoring deletion of file Flask_Login.egg-info/PKG-INFO, use --include-removal to override dpkg-source: warning: ignoring deletion of file Flask_Login.egg-info/requires.txt, use --include-removal to override dpkg-source: warning: ignoring deletion of file Flask_Login.egg-info/SOURCES.txt, use --include-removal to override dpkg-source: warning: ignoring deletion of file Flask_Login.egg-info/top_level.txt, use --include-removal to override dpkg-source: warning: ignoring deletion of file Flask_Login.egg-info/version_info.json, use --include-removal to override dpkg-source: info: building flask-login in flask-login_0.3.2-1.debian.tar.xz dpkg-source: info: building flask-login in flask-login_0.3.2-1.dsc dpkg-genchanges -S -sa >../flask-login_0.3.2-1_source.changes dpkg-genchanges: including full source code in upload dpkg-source --after-build flask-login-0.3.2 dpkg-buildpackage: full upload (original source is included) dpkg-source: warning: extracting unsigned source package (flask-login_0.3.2-1.dsc) dpkg-source: info: extracting flask-login in flask-login-0.3.2 dpkg-source: info: unpacking flask-login_0.3.2.orig.tar.gz dpkg-source: info: unpacking flask-login_0.3.2-1.debian.tar.xz

2 1

Data centric standard for distributing Python modules
by anatoly techtonik 22 Oct '15

22 Oct '15

I see a lot of talks about setuptools/no setuptools/distutils and let me (as usual) state my opinion that all of that is wrong. Instead of discussing who will write the code (which is essentially the root problem) how about just setting the data processing requirements and cover them with tests and stories? Ship supporting infrastructure that allows people from Fedora, Debian, Anaconda see HOW DATA about packages is managed and adopt their tools to support Python style. This will save us all a lot of effort in maintaining the stuff between different systems. As you may see - PEPs are good, but not effective. The testsuite that you can download and test your tools against - that would be awesome thing both for tool development and for LEARNING Python. Why learning is here? Just a simple data definition is not enough - you need to understand why the data structure is there in the first place - for what story it is made for? And when you have the story and the data - it is everything that is needed for a practical Python lesson about how to process files and data formats. Stories are scenarios. My usual scenario is: - install files from a package into local python When you think in data centric way, do you need to know the files that you need to copy from a package into local Python. Yes. But do you need a data format for that into? If you're using wheels - no. Because there are no needless flies. If you're using .rpm? Yes, you need to parse and exclude .spec files etc. Every author may choose their own input - we just define the common output. - uninstall files from local python Here you need the list of files. Where do you get it from? >From some custom specific format located in a directory named after some complicated calculated that is separated over separate PEPs that requires a separate module to calculate. You see. It is where everybody in a trap of using setuptools, distutils etc. I just want to query static data - "give me all files for this package" or "give me all packages with this name so that I can choose specific versions". What should be the data layout to allow that? - create isolated virtualenv from existing files I am pretty sure it is not possible now and everybody is redownloading the stuff again and again to create new environment. - check all my installed modules for security issues - check all my virtualenvs for security issues The last part requires a place for virtualenvs to register their location, so that you can read their modules and give security warning about them too. With test coverage (and beautiful and easy runner) this will allow people to write final tools in a fun way. And if people have fun, they usually create better things than those who are forced to do their thing for a living. And it will be more fun if they can easily extend tests for new stories. -- anatoly t.

1 0

free project idea
by Daniel Holth 21 Oct '15

21 Oct '15

Take this setup.py that implements setup requirements (requirements needed for setup.py itself to run) https://bitbucket.org/dholth/setup-requires/src/tip/setup.py Require your new build system using that mechanism, by mentioning it in setup.cfg Then instead of calling setup() from setuptools at the bottom, emulate the setup.py build system interface documented at http://pip.readthedocs.org/en/stable/reference/pip_install/#build-system-in… to invoke new build system. Check to make sure whether pip accepts .dist-info when calling the egg_info command. Then write a command for new build system that adds the shim setup.py to an sdist. Now you have a setup.py that can download and install new-build-system and interoperate with pip, without having to change pip at all.

2 1

red, green, refactor OR red, green, new feature?
by Thomas Güttler 21 Oct '15

21 Oct '15

Hi distutils friends, which workflow do you prefer? 1. red (test fails) 2. green (test and code works) 3. refactor (clean up, deprecate old stuff) OR 1. red 2. green 3. new feature. I, like most other programmers, love new features and often do workflow2 :-) But thinks get messy in the long run. This is the birds eye view of a particular problem I have. But the particular problem is not that important. Are you willing to clean up and deprecate old stuff? Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/

3 2

Where should I put tests when packaging python modules?
by Thomas Güttler 17 Oct '15

17 Oct '15

Hi, Where should I put tests when packaging python modules? I want a "cowpath", an "obvious way" Dear experts, please decide: inside the module like this answer: http://stackoverflow.com/questions/5341006/where-should-i-put-tests-when-pa… XOR outside the module like this: https://github.com/pypa/sampleproject/tree/master/tests I think there is no need to hurry. Let's wait one week, and then check which one is preferred. Regards, Thomas Güttler -- http://www.thomas-guettler.de/

21 88

Re: [Distutils] PyPi not allowing duplicate filenames
by Chris Barker 15 Oct '15

15 Oct '15

On Wed, Oct 14, 2015 at 9:56 AM, Dave Forgac <tylerdave(a)tylerdave.com> wrote: > This was discussed recently here: > https://github.com/pypa/packaging-problems/issues/74 > and on this list at other times. Though the above issue was pretty focused on restoring a deleted file without any changes -- which seems like a no-brainer to me, as long as someone wants to put the effort into the infrastructure. (the soft delete option seems like a good idea to me). But I'm talking about the cases of "whoops! I really wish I hadn't uploaded that one". We can improve the tooling (some discussion on this in this thread right now...), but people are people and some of us are stupid and/or careless. So this WILL happen. And it just seems pedantic to say: hey -- you've already put that one there -- maybe even two minutes ago, so there is NO WAY to fix your mistake. If it happens quickly, then no one has downloaded it, it hasn't made its way to the mirrors, etc... And again -- we are all adults here: if you as the package maintainer want to do somethign that is sonfusing to your users, is it up to PyPi to never let you do that? (I think it IS up to PyPi to strongly recommend that you don't -- i.e. make it hard to do, and impossible to do thoughtlessly) On Wed, Oct 14, 2015 at 10:00 AM, Jeremy Stanley <fungi(a)yuggoth.org> wrote: > > You should have to jump through all sorts of hoops, and make it > > really clear that it is a BAD IDEA in the general case, but it'd > > be good to have it be possible. > [...] > > It used to be possible. Was it also easy to do without very careful consideration? Or were the hoops I mentioned in place? I can't find it right now, but I think someone in this thread suggested a "staging area", so we could essentially do a trail run: upload to PyPi, tell a few trusted friends about it, have them test it, then, and only then, push it to the official channel. Maybe the infrastructure for that would be more work than it's worth, but I like it. This would fit into the basic principle that you should always be able to test something in as close the final environment as possible, before you commit to it. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

7 14

warning about potential problem for wheels
by Laura Creighton 14 Oct '15

14 Oct '15

(note, I currently don't have mail delivery on for distutils. I could change this, but right now I don't think I have a lot to contribute. This is just a warning). If you have old windows hardware, which does not support SSE2, and windows 7, you can build your own python 3.5. This will work. But wheels won't, you will need to build them from source as well. see: http://bugs.python.org/issue25361 This means that wheels could start failing. It would be good if the wheels themselves could detect this problem and protest in a reasonable fashion, but I have no idea if this is possible. In any case, I thought you needed to know. Laura

9 22