scikits developement conventions ?
Hi, Now that scikits is finally in place, I would like to know what the "conventions" are for development. The only webpage I found for info is the following: https://projects.scipy.org/scipy/scikits/. Is that right ? My questions are: - who has write access to the repository ? - how does one create a new project (people with write access can create a project ?) cheers, David
David Cournapeau wrote:
Hi,
Now that scikits is finally in place, I would like to know what the "conventions" are for development. The only webpage I found for info is the following: https://projects.scipy.org/scipy/scikits/. Is that right ?
That's it at the moment. You can also look at my responses in the thread "mlabwrap scikit". I'll try to flesh out the page this week.
My questions are: - who has write access to the repository ?
Ask Jeff Strunk <jstrunk at enthought.com> for access. Tell him I said to give you access.
- how does one create a new project (people with write access can create a project ?)
svn add, essentially. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On 4/24/07, Robert Kern <robert.kern@gmail.com> wrote:
David Cournapeau wrote:
Now that scikits is finally in place, I would like to know what the "conventions" are for development. The only webpage I found for info is the following: https://projects.scipy.org/scipy/scikits/. Is that right
?
That's it at the moment. You can also look at my responses in the thread "mlabwrap scikit". I'll try to flesh out the page this week.
Last week Alexander Schmolck and I started converting mlabwrap to the be the first scikits project. We ran short on time and didn't get everything finished. I will be adding the project to svn later this week. Alex also started working on the distilling the scipy-dev threads on the wiki page. Once Robert fleshes it out a little better, it should make more sense. -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/
Hi everyone, As the author of the first (if yet unreleased) scikit, I've tried to put some info for prospective scikit authors on <http://projects.scipy.org/scipy/scikits/wiki/WikiStart> that tries to condense down the most important points for those who are not yet familiar with the required infrastructure like setuptools (which is fairly rich) etc. to make things a little less overwhelming. A few things also still need clarification. As I'm myself just learning to get to grips with various of these things, so this page could use some attention from someone more knowledgable than me. Here are a few specific points (search for FIXME/XXX in the document) 0. Blurp: What sort of things are suitable for a scikit project, what requirements must be met by projects or authors to qualify (this should also include something on acceptable/desirable[1] licenses, dependencies, code quality etc.) and what the overall aims and immediate future for scikits look like. I'm not really the person to write that so I just wrote something brief and vague. 1. Trac: I'm not really that familiar with trac, but I assume some sorts of conventions/infrastructure will need to be established in order for logically independent projects to use the same trac effectively. I guess one first step might be to use ``trac-admin`` to create each scikit (i.e. mlabwrap) as a component? 2. Homepage/Wiki/Docstandards etc. info. Hmm, it occurs to me that a link to coding standards might also be a good idea... 3. Versioning: I think it would be nice if there were a standard way how people version their scikits, both conventions-wise as well as how the process is automated as far as is possible. I've outlined a setuptools-based solution in preference to one using numpy.distutils docs pretty opaque[1]. 4. Testing: looks like I'll need to write this some other time... Please fix directly as you see fit or post here if you have suggestions or want to discuss it. thanks, 'as p.s. The transfer of the mlabwrap svn to scipy org has been delayed, otherwise I'd have added some links to the mlabwrap code to. Footnotes: [1] From looking at <http://www.scipy.org/Documentation/numpy_distutils>, it's pretty difficult to figure out what it's good for -- there is no rationale who might want to use it and why (see below for an attempt), instead it starts out by describing the scipy structure and requirements for scipy packages Evidently some people who are not interested in writing scipy packages might also benefit from numpy.distutils (scikits authors for example), so it'd would really help to have a short list of features that numpy.distutils has over plain distutils and setuptools and to whom they might be relevant. The second obvious question is how numpy.distutils relates to/plays along with setuptools. Here is an attempt to what the numpy.distutils could say at the beginning concerning possible candidates for packages that should use numpy.distutils: - all scipy packages (?) - packages that need fortran extensions - packages that requires subpackages (but setuptools should take care of that too?) - packages that want certain conveniences such as non manual version-info-support and other code-generation I think setuptools should also be explicitly mentioned.
Alexander Schmolck wrote:
Hi everyone,
As the author of the first (if yet unreleased) scikit, I've tried to put some info for prospective scikit authors on
<http://projects.scipy.org/scipy/scikits/wiki/WikiStart>
that tries to condense down the most important points for those who are not yet familiar with the required infrastructure like setuptools (which is fairly rich) etc. to make things a little less overwhelming. A few things also still need clarification. As I'm myself just learning to get to grips with various of these things, so this page could use some attention from someone more knowledgable than me.
Here are a few specific points (search for FIXME/XXX in the document)
0. Blurp: What sort of things are suitable for a scikit project, what requirements must be met by projects or authors to qualify (this should also include something on acceptable/desirable[1] licenses, dependencies, code quality etc.) and what the overall aims and immediate future for scikits look like. I'm not really the person to write that so I just wrote something brief and vague.
1. Trac: I'm not really that familiar with trac, but I assume some sorts of conventions/infrastructure will need to be established in order for logically independent projects to use the same trac effectively. I guess one first step might be to use ``trac-admin`` to create each scikit (i.e. mlabwrap) as a component?
2. Homepage/Wiki/Docstandards etc. info.
Hmm, it occurs to me that a link to coding standards might also be a good idea...
3. Versioning: I think it would be nice if there were a standard way how people version their scikits, both conventions-wise as well as how the process is automated as far as is possible. I've outlined a setuptools-based solution in preference to one using numpy.distutils docs pretty opaque[1].
4. Testing: looks like I'll need to write this some other time...
Please fix directly as you see fit or post here if you have suggestions or want to discuss it.
One thing which may need to be changed is the license requirement. I thought one of the point of scikits was to include code useful for scipy community, but which cannot be licensed to BDS/MIT. For example, including GPL code, or package which depends on LGPL libraries. malabwrap for that matter depends on proprietary libraries to work... Once this issue is solved, I would be happy to give two toolboxes of my own for audio signal processing (released under the BSD, but depending on LGPL libraries). David
On 5/16/07, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
One thing which may need to be changed is the license requirement. I thought one of the point of scikits was to include code useful for scipy community, but which cannot be licensed to BDS/MIT. For example, including GPL code, or package which depends on LGPL libraries. malabwrap for that matter depends on proprietary libraries to work...
Thanks to Alex for putting together this information. I don't have the time to respond to all his points/questions, but I wanted to quickly respond to David's comments about licensing. Originally, I assumed that *all* the scikit code would be released under the same license (ie., the revised BSD license like all the rest of scipy). That would certainly be my preference. My understanding was that the basic idea behind scikits was simply to provide a modular mechanism for distributing "scipy" packages under one namespace. There are several arguments for using the same license for all the packages. First, I think that it would confusing for users importing "from scikits import foo, bar" to find that foo is released under one license and bar is released under another. Second while we want to keep dependencies between scikits packages to a minimum, there will inevitably be a need for code from one package to be used in another. This may lead to code duplication if the needed functionality already exists in a scikit's package, but is released under the GPL when BSD code is needed. Third, it is reasonable to imagine that at some point, someone will want to refactor the various scikit's packages. During a major refactoring, it may be desirable to move code from one package to another or even from scikits to scipy. This will be difficult if it is necessary to keep track of what license each piece of code is released under. Basically, I think that opening up the possibility for each package to be released under its own license is going to make it difficult to maintain some unity to the scipy namespace. I also don't know whether there is a need to include non-BSD code in the scikit's namespace. We could easily create links to non-BSD code on the wikipages or even create a new namespace for code released under different licenses. I would be very interested to hear arguments for why non-BSD code needs to be available under the scikit's namespace. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/
Jarrod Millman wrote:
On 5/16/07, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
One thing which may need to be changed is the license requirement. I thought one of the point of scikits was to include code useful for scipy community, but which cannot be licensed to BDS/MIT. For example, including GPL code, or package which depends on LGPL libraries. malabwrap for that matter depends on proprietary libraries to work...
Thanks to Alex for putting together this information. I don't have the time to respond to all his points/questions, but I wanted to quickly respond to David's comments about licensing.
Originally, I assumed that *all* the scikit code would be released under the same license (ie., the revised BSD license like all the rest of scipy). That would certainly be my preference. My understanding was that the basic idea behind scikits was simply to provide a modular mechanism for distributing "scipy" packages under one namespace. Ok, my understanding was different on this issue. I thought scikits and scipy had different purpose, and that one of scikits' goal was to enable code depending on GPL code. I thought scipy modularization was a totally different issue.
There are several arguments for using the same license for all the packages. First, I think that it would confusing for users importing "from scikits import foo, bar" to find that foo is released under one license and bar is released under another. Second while we want to keep dependencies between scikits packages to a minimum, there will inevitably be a need for code from one package to be used in another. This may lead to code duplication if the needed functionality already exists in a scikit's package, but is released under the GPL when BSD code is needed. Third, it is reasonable to imagine that at some point, someone will want to refactor the various scikit's packages. During a major refactoring, it may be desirable to move code from one package to another or even from scikits to scipy. This will be difficult if it is necessary to keep track of what license each piece of code is released under.
Basically, I think that opening up the possibility for each package to be released under its own license is going to make it difficult to maintain some unity to the scipy namespace. I also don't know whether there is a need to include non-BSD code in the scikit's namespace.
For example, my package pyaudiolab is licensed under the BSD, but depends on LGPL libraries. Technically, it is a BSD package, but it is unusable without non BSD code. This was given as a reason why pyaudiolab could not be included in scipy, but could be included in scikits. There is also the problem of GPL libraries (qt, pyqt come to mind): I don't think you can use them in BSD code, right ? But I guess this becomes the grey area of derivative work. Can I use GPL libraries in non GPL python if I access the library though ctypes ? If there is a python package under the GPL, and I import it in my own package, do I have to use the GPL ? David
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes:
Ok, my understanding was different on this issue. I thought scikits and scipy had different purpose, and that one of scikits' goal was to enable code depending on GPL code. I thought scipy modularization was a totally different issue.
Well, this is something the official developers of scipy have to make their minds up about; but personally I'm not sure allowing GPLed code in anywhere is a good idea -- IMO there's enough viral scientific computing code out there already. Allowing LGPL or LGPL-dependent code in scikits is a different matter; I think BSD/MIT-style should be encouraged but I think it would be useful for scikits to be able to include modules that wrap or use a LGPLed 3rd party lib for which no suitable freer alternatives exist.
There is also the problem of GPL libraries (qt, pyqt come to mind): I don't think you can use them in BSD code, right ? But I guess this becomes the grey area of derivative work.
No, not really. The FSF did its best to make sure that only GPLed code can use GPLed code -- that's the whole point.
Can I use GPL libraries in non GPL python if I access the library though ctypes ?
I don't think so.
If there is a python package under the GPL, and I import it in my own package, do I have to use the GPL ?
Yes. The only exception might be if your package can *optionally* use something that's GPLed (in which case it would still have to be GPL compatible), but even then you might find yourself on shaky ground (google clisp readline GPL). IANAL etc., but unless you are, I think you're best off assuming that GPLed code is off-limits unless you want to GPL your code too, because the FSF has hired good lawyers to make it that way. cheers, 'as
Alexander Schmolck <a.schmolck@gmx.net> writes:
Can I use GPL libraries in non GPL python if I access the library though ctypes ?
I don't think so.
Just to clarify: your python code will have to be GPLed (at least that's what I strongly assume) but python license is not a hindrance (its GPL-compatbily, which essentially means one can relicense python under the GPL). 'as
On Mon, May 21, 2007 at 09:42:25AM +0100, Alexander Schmolck wrote:
There is also the problem of GPL libraries (qt, pyqt come to mind): I don't think you can use them in BSD code, right ? But I guess this becomes the grey area of derivative work.
No, not really. The FSF did its best to make sure that only GPLed code can use GPLed code -- that's the whole point.
That's not true, strictly speaking. You can still license *your* part of the software under the modified BSD license. From http://www.gnu.org/licenses/gpl-faq.html#CombinePublicDomainWithGPL """ If a program combines public-domain code with GPL-covered code, can I take the public-domain part and use it as public domain code? You can do that, if you can figure out which part is the public domain part and separate it from the rest. If code was put in the public domain by its developer, it is in the public domain no matter where it has been. """ The combined product, however, must be distributed under terms of the GPL. Since users of Python and Scipy expect BSD-compatible licenses, it might be a good idea to have scikits print their license terms on import. Regards Stéfan
Stefan van der Walt <stefan@sun.ac.za> writes:
On Mon, May 21, 2007 at 09:42:25AM +0100, Alexander Schmolck wrote:
There is also the problem of GPL libraries (qt, pyqt come to mind): I don't think you can use them in BSD code, right ? But I guess this becomes the grey area of derivative work.
No, not really. The FSF did its best to make sure that only GPLed code can use GPLed code -- that's the whole point.
That's not true, strictly speaking.
I think it is.
You can still license *your* part of the software under the modified BSD license.
My understanding is you can't if it depends on the GPLed code. You'd have to write a non-GPL'ed drop-in replacement first. 'as
Alexander Schmolck wrote:
Stefan van der Walt <stefan@sun.ac.za> writes:
You can still license *your* part of the software under the modified BSD license.
My understanding is you can't if it depends on the GPLed code. You'd have to write a non-GPL'ed drop-in replacement first.
Your understanding is incorrect. You can combine code under a GPL-compatible but non-GPL license with GPLed code. The work as a whole must be distributed under the terms of the GPL. The readline module in Python, for example, is distributed under the PSF license just like the rest of Python. There is no drop-in replacement for libreadline (I don't believe libedit has been a drop-in replacement for many years, now). The non-GPLed bit *must* be GPL-compatible, but it does not have to be GPLed. There are explicit statements from Richard Stallman to that effect. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Robert Kern <robert.kern@gmail.com> writes:
Alexander Schmolck wrote:
Stefan van der Walt <stefan@sun.ac.za> writes:
You can still license *your* part of the software under the modified BSD license.
My understanding is you can't if it depends on the GPLed code. You'd have to write a non-GPL'ed drop-in replacement first.
Your understanding is incorrect. You can combine code under a GPL-compatible but non-GPL license with GPLed code. The work as a whole must be distributed under the terms of the GPL. The readline module in Python, for example, is distributed under the PSF license just like the rest of Python. There is no drop-in replacement for libreadline (I don't believe libedit has been a drop-in replacement for many years, now).
Thanks for the clarification. The GPL is weaker then than I thought then, because that means you can rewrite a GPL'ed program or library piecewise thus eventually "liberating" it from the GPL. I am surprised that I haven't heard of anyone doing that -- apart from the likely bad karma, it seems like a reasonable strategy in certain cases... maybe because it's difficult to argue that it would be really clean-room... 'as
Alexander Schmolck wrote:
Robert Kern <robert.kern@gmail.com> writes:
Stefan van der Walt <stefan@sun.ac.za> writes:
You can still license *your* part of the software under the modified BSD license. My understanding is you can't if it depends on the GPLed code. You'd have to write a non-GPL'ed drop-in replacement first. Your understanding is incorrect. You can combine code under a GPL-compatible but non-GPL license with GPLed code. The work as a whole must be distributed under
Alexander Schmolck wrote: the terms of the GPL. The readline module in Python, for example, is distributed under the PSF license just like the rest of Python. There is no drop-in replacement for libreadline (I don't believe libedit has been a drop-in replacement for many years, now).
Thanks for the clarification. The GPL is weaker then than I thought then, because that means you can rewrite a GPL'ed program or library piecewise thus eventually "liberating" it from the GPL. I am surprised that I haven't heard of anyone doing that -- apart from the likely bad karma, it seems like a reasonable strategy in certain cases... maybe because it's difficult to argue that it would be really clean-room...
It wouldn't be. That's why you haven't heard of anyone doing it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Robert Kern <robert.kern@gmail.com> writes:
It wouldn't be. That's why you haven't heard of anyone doing it.
Why not, if your eventual goal is BSD anyway? I think if I had to do a BSD'd netlab lookalike and intended to use the library for internal stuff in the meantime, I think I'd rather start out with netlab than from scratch. 'as
Alexander Schmolck wrote:
Robert Kern <robert.kern@gmail.com> writes:
It wouldn't be. That's why you haven't heard of anyone doing it.
Why not, if your eventual goal is BSD anyway? I think if I had to do a BSD'd netlab lookalike and intended to use the library for internal stuff in the meantime, I think I'd rather start out with netlab than from scratch.
Pardon my ellipses. "It wouldn't be [clean room]." -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Hi all, I think scikits should be represented as a bunch of software with OSI-approved licenses. Each routine should inform which license it has, by including the information in routine description, and/or by text or graphical output while working, and/or somehow else. Of course, there could be devision like modules "scikitsbsd" and "scikitsgpl", but what for other dozens of OSI-approved licenses? For example, take a look at PSF page: http://www.python.org/psf/contrib/ they require either * *Academic Free License v. 2.1 or Apache License, Version 2.0 Of course, other licenses can appear in future, and one software package, for example lpsolve, can change his license from LGPL to BSD (or vise versa) - so how do you intend to handle the situation? //just my 15 cents Regards, D. * * Jarrod Millman wrote:
On 5/16/07, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
One thing which may need to be changed is the license requirement. I thought one of the point of scikits was to include code useful for scipy community, but which cannot be licensed to BDS/MIT. For example, including GPL code, or package which depends on LGPL libraries. malabwrap for that matter depends on proprietary libraries to work...
Thanks to Alex for putting together this information. I don't have the time to respond to all his points/questions, but I wanted to quickly respond to David's comments about licensing.
Originally, I assumed that *all* the scikit code would be released under the same license (ie., the revised BSD license like all the rest of scipy). That would certainly be my preference. My understanding was that the basic idea behind scikits was simply to provide a modular mechanism for distributing "scipy" packages under one namespace.
There are several arguments for using the same license for all the packages. First, I think that it would confusing for users importing "from scikits import foo, bar" to find that foo is released under one license and bar is released under another. Second while we want to keep dependencies between scikits packages to a minimum, there will inevitably be a need for code from one package to be used in another. This may lead to code duplication if the needed functionality already exists in a scikit's package, but is released under the GPL when BSD code is needed. Third, it is reasonable to imagine that at some point, someone will want to refactor the various scikit's packages. During a major refactoring, it may be desirable to move code from one package to another or even from scikits to scipy. This will be difficult if it is necessary to keep track of what license each piece of code is released under.
Basically, I think that opening up the possibility for each package to be released under its own license is going to make it difficult to maintain some unity to the scipy namespace. I also don't know whether there is a need to include non-BSD code in the scikit's namespace. We could easily create links to non-BSD code on the wikipages or even create a new namespace for code released under different licenses. I would be very interested to hear arguments for why non-BSD code needs to be available under the scikit's namespace.
Thanks,
Jarrod Millman wrote:
Originally, I assumed that *all* the scikit code would be released under the same license (ie., the revised BSD license like all the rest of scipy). That would certainly be my preference. My understanding was that the basic idea behind scikits was simply to provide a modular mechanism for distributing "scipy" packages under one namespace.
This is not the case. An explicit goal was for each package to have its own license such that we can provide project infrastructure for projects that have more restrictive licenses than scipy. It is *not* the intention to move scipy.linalg, etc. over to scikits. Nothing currently in scipy (except maybe some of the currently sandboxed projects) will move over to scikits.
There are several arguments for using the same license for all the packages. First, I think that it would confusing for users importing "from scikits import foo, bar" to find that foo is released under one license and bar is released under another. Second while we want to keep dependencies between scikits packages to a minimum, there will inevitably be a need for code from one package to be used in another. This may lead to code duplication if the needed functionality already exists in a scikit's package, but is released under the GPL when BSD code is needed. Third, it is reasonable to imagine that at some point, someone will want to refactor the various scikit's packages. During a major refactoring, it may be desirable to move code from one package to another or even from scikits to scipy. This will be difficult if it is necessary to keep track of what license each piece of code is released under.
I don't think that will be difficult at all. You just have to stop thinking of "scikits" as a project. It's not. It's just a namespace for other projects.
Basically, I think that opening up the possibility for each package to be released under its own license is going to make it difficult to maintain some unity to the scipy namespace. I also don't know whether there is a need to include non-BSD code in the scikit's namespace. We could easily create links to non-BSD code on the wikipages or even create a new namespace for code released under different licenses. I would be very interested to hear arguments for why non-BSD code needs to be available under the scikit's namespace.
There is a need. It was, in fact, the *largest* motivation for starting scikits. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
David Cournapeau wrote:
One thing which may need to be changed is the license requirement. I thought one of the point of scikits was to include code useful for scipy community, but which cannot be licensed to BDS/MIT. For example, including GPL code, or package which depends on LGPL libraries. malabwrap for that matter depends on proprietary libraries to work...
Once this issue is solved, I would be happy to give two toolboxes of my own for audio signal processing (released under the BSD, but depending on LGPL libraries).
This is the official word on scikits licensing: scikits packages are free to choose their own open source license. The license should be officially OSI approved. We will allow packages to contain code with licenses that, in our judgment, comply with the Open Source Definition but have not gone through the approval process. This is to allow us to adopt old code with permissive licenses. The package itself, though, should use a well-known OSI-approved license. Packages depending on proprietary code are discouraged, but we will allow some packages on a case-by-case basis. Primarily, only packages which depend on proprietary code in order to interact with a proprietary system will be allowed. Wrappers for proprietary *libraries* most likely won't. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Robert Kern <robert.kern@gmail.com> writes:
This is the official word on scikits licensing:
OK, I think this bit should just go verbatim on the scikits wiki:
scikits packages are free to choose their own open source license. The license should be officially OSI approved. We will allow packages to contain code with licenses that, in our judgment, comply with the Open Source Definition but have not gone through the approval process. This is to allow us to adopt old code with permissive licenses. The package itself, though, should use a well-known OSI-approved license.
I think a recommendation for packages whose authors don't have any strong preferences or that do not have other license constraints would be good to give (BSD? Anything else that's just as good?), because this will reduce unintended license incompatibilities between packages (not all OSI licenses are compatible, obviously). cheers, 'as
Alexander Schmolck wrote:
Robert Kern <robert.kern@gmail.com> writes:
This is the official word on scikits licensing:
OK, I think this bit should just go verbatim on the scikits wiki:
scikits packages are free to choose their own open source license. The license should be officially OSI approved. We will allow packages to contain code with licenses that, in our judgment, comply with the Open Source Definition but have not gone through the approval process. This is to allow us to adopt old code with permissive licenses. The package itself, though, should use a well-known OSI-approved license.
I think a recommendation for packages whose authors don't have any strong preferences or that do not have other license constraints would be good to give (BSD? Anything else that's just as good?), because this will reduce unintended license incompatibilities between packages (not all OSI licenses are compatible, obviously).
""" The recommended license for scikits projects is the (new) BSD license. http://www.opensource.org/licenses/bsd-license.html """ Unofficially, for David, I do recommend against using the BSD license for wrappers of LGPLed or GPLed libraries. It just confuses matters. There probably isn't much code in the wrapper that can be usefully separated from the library. For such bits, people can simply ask you for a new license on the little bits of code. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Robert Kern <robert.kern@gmail.com> writes:
Alexander Schmolck wrote:
Robert Kern <robert.kern@gmail.com> writes:
This is the official word on scikits licensing:
OK, I think this bit should just go verbatim on the scikits wiki:
scikits packages are free to choose their own open source license. The license should be officially OSI approved. We will allow packages to contain code with licenses that, in our judgment, comply with the Open Source Definition but have not gone through the approval process. This is to allow us to adopt old code with permissive licenses. The package itself, though, should use a well-known OSI-approved license.
I think a recommendation for packages whose authors don't have any strong preferences or that do not have other license constraints would be good to give (BSD? Anything else that's just as good?), because this will reduce unintended license incompatibilities between packages (not all OSI licenses are compatible, obviously).
""" The recommended license for scikits projects is the (new) BSD license. http://www.opensource.org/licenses/bsd-license.html """
Thanks -- I have updated <http://projects.scipy.org/scipy/scikits/wiki/WikiStart> accordingly. OK, as far as I'm concerned the scikits wiki page is almost usable now and are the last 2 remaining vital pieces of information that are missing are: 1. Homepage/Download urls I assume the .eggs should end up on cheeseshop and the project homepage will just be the ../scikits/wiki/<PROJECT-NAME>? E.g mlabwrap's setup would include: setup(... url="http://projects.scipy.org/scipy/scikits/wiki/mlabwrap", download_url="http://cheeseshop.python.org/pypi/mlabwrap/" ...) 2. Trac: How should individual scikits use the common trac? Does every scikit become a component? If so, whom does one have to bug to get the component created? Also, provided the rest looks OK to you, I'll remove the "WARNING THIS INFORMATION IS NOT RELIABLE YET" once these two points are addressed. cheers, 'as
Robert Kern wrote:
""" The recommended license for scikits projects is the (new) BSD license. http://www.opensource.org/licenses/bsd-license.html """
Unofficially, for David, I do recommend against using the BSD license for wrappers of LGPLed or GPLed libraries. It just confuses matters. There probably isn't much code in the wrapper that can be usefully separated from the library. For such bits, people can simply ask you for a new license on the little bits of code.
So you mean relicensing under the GPL ? I have to confess that I myself much prefer the GPL to BSD anyway, and the only reason I released the code under BSD was to make it easier to integrate with scipy (that project was started before scikits). David
David Cournapeau wrote:
Robert Kern wrote:
Unofficially, for David, I do recommend against using the BSD license for wrappers of LGPLed or GPLed libraries. It just confuses matters. There probably isn't much code in the wrapper that can be usefully separated from the library. For such bits, people can simply ask you for a new license on the little bits of code. So you mean relicensing under the GPL ?
I recommend making wrappers available under the license of the underlying library if it's one of the standard ones. If it's GPL, then GPL. If it's LGPL, then LGPL. BSD, then BSD. Some ad hoc, but permissive license, then BSD. You said that the underlying library was LGPLed, so I recommend the LGPL in that case.
I have to confess that I myself much prefer the GPL to BSD anyway, and the only reason I released the code under BSD was to make it easier to integrate with scipy (that project was started before scikits).
Well, since that wouldn't have worked anyways, there's not much point in keeping the wrappers as BSD. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Alexander Schmolck wrote:
Hi everyone,
As the author of the first (if yet unreleased) scikit, I've tried to put some info for prospective scikit authors on
<http://projects.scipy.org/scipy/scikits/wiki/WikiStart>
that tries to condense down the most important points for those who are not yet familiar with the required infrastructure like setuptools (which is fairly rich) etc. to make things a little less overwhelming. A few things also still need clarification. As I'm myself just learning to get to grips with various of these things, so this page could use some attention from someone more knowledgable than me.
Here are a few specific points (search for FIXME/XXX in the document)
0. Blurp: What sort of things are suitable for a scikit project, what requirements must be met by projects or authors to qualify (this should also include something on acceptable/desirable[1] licenses, dependencies, code quality etc.) and what the overall aims and immediate future for scikits look like. I'm not really the person to write that so I just wrote something brief and vague.
1. Trac: I'm not really that familiar with trac, but I assume some sorts of conventions/infrastructure will need to be established in order for logically independent projects to use the same trac effectively. I guess one first step might be to use ``trac-admin`` to create each scikit (i.e. mlabwrap) as a component?
2. Homepage/Wiki/Docstandards etc. info.
Hmm, it occurs to me that a link to coding standards might also be a good idea...
3. Versioning: I think it would be nice if there were a standard way how people version their scikits, both conventions-wise as well as how the process is automated as far as is possible. I've outlined a setuptools-based solution in preference to one using numpy.distutils docs pretty opaque[1].
4. Testing: looks like I'll need to write this some other time...
Please fix directly as you see fit or post here if you have suggestions or want to discuss it.
I was thinking, why not providing a "fake" scikits package as an example ? I have mostly converted one project to scikits/setuptools, and the info I think may be useful are: * how to use numpy.distutils (in particular, system_info classes) ? * I cannot make the proposed versioning scheme work, because pkg_resources.require('scikits') fails, even though I installed it (I can import it). I guess I am doing something wrong in the scikits namespace, or for the installation, but I don't know what. * more generally, how to convert a distutils pkg to setuptools. cheers, David
thanks,
'as
p.s. The transfer of the mlabwrap svn to scipy org has been delayed, otherwise I'd have added some links to the mlabwrap code to.
Footnotes: [1] From looking at <http://www.scipy.org/Documentation/numpy_distutils>, it's pretty difficult to figure out what it's good for -- there is no rationale who might want to use it and why (see below for an attempt), instead it starts out by describing the scipy structure and requirements for scipy packages Evidently some people who are not interested in writing scipy packages might also benefit from numpy.distutils (scikits authors for example), so it'd would really help to have a short list of features that numpy.distutils has over plain distutils and setuptools and to whom they might be relevant. The second obvious question is how numpy.distutils relates to/plays along with setuptools.
Here is an attempt to what the numpy.distutils could say at the beginning concerning possible candidates for packages that should use numpy.distutils:
- all scipy packages (?) - packages that need fortran extensions - packages that requires subpackages (but setuptools should take care of that too?) - packages that want certain conveniences such as non manual version-info-support and other code-generation
I think setuptools should also be explicitly mentioned. _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes:
I was thinking, why not providing a "fake" scikits package as an example ?
There definitely should be some package that can serve as an example and the wiki-page should link to its svn-setuptools.py (one of the reasons I'd like to get mlabwrap into the official scikits svn soon). Fake packages share the same disadvantage over real code as comments and docs, though -- there's nothing to ensure they're up to date and accurate -- which is why I'd favor using a real package, unless there isn't anything that's both suitably simple and not completely trivial. I think it might even be best to eventually have 2 or 3 types of example packages: 1. A data-only package 2. A python-only package 3. Something that builds a C-extension We should of course makes sure that the setup.py files of these packages do really implement 'best practices'; once these setup.py files look reasonable we could even see if people in distutils-sig might be willing to give them a quick glance.
I have mostly converted one project to scikits/setuptools, and the info I think may be useful are: * how to use numpy.distutils (in particular, system_info classes) ?
Why do you want to use numpy.distutils? Doesn't setuptools provide the functionality you want (it might not, but I'd personally just avoid using numpy.distutils for everything for which it isn't strictly required)?
* I cannot make the proposed versioning scheme work, because pkg_resources.require('scikits') fails, even though I installed it (I can import it).
Hmm, why are your trying to require('scikits')? Scikits is just a namespace, so it doesn't really make sense to require it, I think (it doesn't make that much sense to import it *either*, but that's just how python's module system works). As I wrote on the wiki-page, you want to do: from pkg_resources import require __version__ = require('scikits.mlabwrap')[0].version substituting your project name -- doesn't that work for you?
I guess I am doing something wrong in the scikits namespace, or for the installation, but I don't know what. * more generally, how to convert a distutils pkg to setuptools.
Changing from ``distutils import`` to from ``setuptools import`` should be enough for a minimal conversion, I think. If that works OK for you I suggest we add a subheading 'minimal conversion from a distutils-based install', including the required (``namespace_packages=['scikits']``) and recommended (``zip_safe=True|False, url='...', download_url='...',test_suite``) additional setuptools flags (unfortunatelly no one has yet expounded on where the 'homepages' of individual scikits are meant to go). cheers, 'as
Alexander Schmolck wrote:
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes:
I was thinking, why not providing a "fake" scikits package as an example ?
There definitely should be some package that can serve as an example and the wiki-page should link to its svn-setuptools.py (one of the reasons I'd like to get mlabwrap into the official scikits svn soon). Fake packages share the same disadvantage over real code as comments and docs, though -- there's nothing to ensure they're up to date and accurate -- which is why I'd favor using a real package, unless there isn't anything that's both suitably simple and not completely trivial.
I think it might even be best to eventually have 2 or 3 types of example packages:
1. A data-only package 2. A python-only package 3. Something that builds a C-extension
We should of course makes sure that the setup.py files of these packages do really implement 'best practices'; once these setup.py files look reasonable we could even see if people in distutils-sig might be willing to give them a quick glance.
I have mostly converted one project to scikits/setuptools, and the info I think may be useful are: * how to use numpy.distutils (in particular, system_info classes) ?
Why do you want to use numpy.distutils? Doesn't setuptools provide the functionality you want (it might not, but I'd personally just avoid using numpy.distutils for everything for which it isn't strictly required)? I am using the system_info scheme from distutils (to retrieve a shared
Well, if we have one example with many options, I guess it should not be too difficult to make it up-to-date. For example, for the package I would like to submit to scikits, I have a makefile which builds the pdf doc using some python scripts, and try to run those scripts to make sure everything is updated. library and its header). I could do without, but that would basically means reimplementing system_info, I guess.
Hmm, why are your trying to require('scikits')? Scikits is just a namespace, so it doesn't really make sense to require it, I think (it doesn't make that much sense to import it *either*, but that's just how python's module system works).
As I wrote on the wiki-page, you want to do:
from pkg_resources import require __version__ = require('scikits.mlabwrap')[0].version
substituting your project name -- doesn't that work for you?
No, it does not. I assumed the problem was with scikits, because I do not have a .egg file for scikits when I install my package (whereas I have a pyaudiolab.egg). But I know nothing about eggs (I do not even know what they are used for). The other problem I have is that python setup.py test does not work either. After some random investigation, it looks like setuptools do not call check_* functions, only test_*. Does that mean I have to use only check_* functions in my tests ? David
The other problem I have is that python setup.py test does not work either. After some random investigation, it looks like setuptools do not call check_* functions, only test_*. Does that mean I have to use only check_* functions in my tests ?
Just a small chime in on that last point -- I don't think there's any reason to use the check_* form of test method names (and I plan to switch mlabwrap's test methods back to test_*). -Brian David
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes:
Alexander Schmolck wrote:
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes: Well, if we have one example with many options, I guess it should not be too difficult to make it up-to-date. For example, for the package I would like to submit to scikits, I have a makefile which builds the pdf doc using some python scripts, and try to run those scripts to make sure everything is updated.
Sorry, I'm not quite clear what you are getting at, so possibly this answer doesn't address your point at all but basically what I was trying to say is that if you use some real package, chances are that unless gets abandonned, someone will be pretty motivated to make sure its install works properly (even if something in setuptools changes) -- more so than if its just a toy for the benefit of *others*.
I have mostly converted one project to scikits/setuptools, and the info I think may be useful are: * how to use numpy.distutils (in particular, system_info classes) ?
Why do you want to use numpy.distutils? Doesn't setuptools provide the functionality you want (it might not, but I'd personally just avoid using numpy.distutils for everything for which it isn't strictly required)? I am using the system_info scheme from distutils (to retrieve a shared library and its header). I could do without, but that would basically means reimplementing system_info, I guess.
That's unfortunate -- I find numpy.distutils pretty impenetrable, and I also tend to believe that pretty much anything that's useful about it ought to eventually end up in setuptools, unless it's really specific to numpy or numerical python code -- IMO numpy having so much general-purpose building (and testing) facilities in numpy is a bad thing and increases the barrier of entry for scikits writers and other contributors of numerical python code unnecessarily, because the learning curve gets steeper and the number of wrong or largely equivalent choices and combinations that won't work grows. Of course if you manage to figure out what parts of numpy.distutils might be helpful for scikits-users, and how to use them, please do share your insights on the wiki -- using functionality from numpy.distutils is certainly better than roling one's own and will likely remain necessary for some time. But I do think that the longer term goal of the scipy-community should be to rely more on setuptools and less on numpy.distutils functionality (as far as non-special-purpose stuff is concerned; support for fortran stuff e.g. might well be special-purpose enough to keep it out of setuptools).
Hmm, why are your trying to require('scikits')? Scikits is just a namespace, so it doesn't really make sense to require it, I think (it doesn't make that much sense to import it *either*, but that's just how python's module system works).
As I wrote on the wiki-page, you want to do:
from pkg_resources import require __version__ = require('scikits.mlabwrap')[0].version
substituting your project name -- doesn't that work for you? No, it does not.
What happens? If your code doesn't have any strange dependencies and you if email it to me (or point me to an url with a tgz), I could try and have a brief look at it, if you want.
I assumed the problem was with scikits, because I do not have a .egg file for scikits when I install my package (whereas I have a pyaudiolab.egg). But I know nothing about eggs (I do not even know what they are used for).
Eggs are like jar files. Essentially they are both a distribution format (like rpm) and an archive format from which you can run code directly.
The other problem I have is that python setup.py test does not work either. After some random investigation, it looks like setuptools do not call check_* functions, only test_*. Does that mean I have to use only check_* functions in my tests ?
[test_* you mean?] Yes, that's what I'd suggest. check_* and bench_* are numpy.testing only -- I've never understood the point of introducing them as additional aliases for 'test*' (which is what unittest uses, by default), it just seems to create unnecessary confusion and incompatiblity without any real benfit. Having said that something like this should also work: # test_pyaudialab.py [...] def test_em(): NumpyTest().run() # or whatever # setup.py [...] test_suite="scikits.pyaudiolab.tests.test_pyaudialab.test_em", [...] cheers, 'as
Alexander Schmolck wrote:
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes:
Alexander Schmolck wrote:
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes: Well, if we have one example with many options, I guess it should not be too difficult to make it up-to-date. For example, for the package I would like to submit to scikits, I have a makefile which builds the pdf doc using some python scripts, and try to run those scripts to make sure everything is updated.
Sorry, I'm not quite clear what you are getting at, so possibly this answer doesn't address your point at all but basically what I was trying to say is that if you use some real package, chances are that unless gets abandonned, someone will be pretty motivated to make sure its install works properly (even if something in setuptools changes) -- more so than if its just a toy for the benefit of *others*. I was just saying that if the only drawback of having a "fake" package is that it is not updated, it should not be too difficult to have something in place to test it automatically.
But anyway, this is a very minor point (real package vs fake package).
That's unfortunate -- I find numpy.distutils pretty impenetrable, and I also tend to believe that pretty much anything that's useful about it ought to eventually end up in setuptools, unless it's really specific to numpy or numerical python code -- IMO numpy having so much general-purpose building (and testing) facilities in numpy is a bad thing and increases the barrier of entry for scikits writers and other contributors of numerical python code unnecessarily, because the learning curve gets steeper and the number of wrong or largely equivalent choices and combinations that won't work grows.
I agree with you here, but that's a consequence of the history of numpy, I guess. Now, I don't find setuptools that much clearer myself. Maybe it is just because I have not used it extensively, but I always found setuptools (and distutils for that matter) quite complicated for anything not trivial. I had to spend 2 hours to convert my project to setuptools, and for the only benefit to have support for namespace. And it still does not work because for some reasons, files automatically included by setuptools for install target are not included when using sdist target, whereas there is no difference between those targets with distutils... And I have not tested it yet on other platforms (I am always afraid of windows :) )
course if you manage to figure out what parts of numpy.distutils might be helpful for scikits-users, and how to use them, please do share your insights on the wiki -- using functionality from numpy.distutils is certainly better than roling one's own and will likely remain necessary for some time. But I do think that the longer term goal of the scipy-community should be to rely more on setuptools and less on numpy.distutils functionality (as far as non-special-purpose stuff is concerned; support for fortran stuff e.g. might well be special-purpose enough to keep it out of setuptools). I don't know much about numpy.distutils, but if you take a quick look at numpy/distutils, it is around 1.9 Mb, of which: - 1 M of tests - 350 ko for fortran (fcompiler) - 300 ko for extending distutils (command) - half of the rest seems pretty specific to numpy too (system_info is, cpuinfo too, other I don't know).
You could say that the modules of numpy/scipy fall into 3 categories wrt distutils: - pure python packages: not difficult to handle with distutils - python packages which use their own C extension: not difficult to handle with distutils. - python packages which depend on C/Fortran code (other libraries): difficult to handle with distutils, if not impossible. In my case, I use numpy.distutils to find a library and its header files. As far as I know, neither distutils or setuptools offer any facility for that.
What happens?
I got the following error when importing the package (from scikits import pyaudiolab) <class 'pkg_resources.DistributionNotFound'>Traceback (most recent call last) /usr/media/boulot/softwares/JapSoft/pykdic/src/<ipython console> in <module>() /home/david/local/lib/python2.5/site-packages/scikits/pyaudiolab/__init__.py in <module>() 2 # Last Change: Wed Jan 31 03:00 PM 2007 J 3 ----> 4 from info import __doc__ 5 6 #from pyaudioio import play as linux_player /home/david/local/lib/python2.5/site-packages/scikits/pyaudiolab/info.py in <module>() 15 License: BSD-style (see LICENSE.txt in main source directory) 16 """ ---> 17 import version as vpy 18 version = vpy.__version__ 19 /home/david/local/lib/python2.5/site-packages/scikits/pyaudiolab/version.py in <module>() 3 4 from pkg_resources import require ----> 5 __version__ = require('scikits.pyaudiolab')[0].version 6 #__version__ = '0.7' 7 /usr/lib/python2.5/site-packages/pkg_resources.py in require(self, *requirements) 583 """ 584 --> 585 needed = self.resolve(parse_requirements(requirements)) 586 587 for dist in needed: /usr/lib/python2.5/site-packages/pkg_resources.py in resolve(self, requirements, env, installer) 481 dist = best[req.key] = env.best_match(req, self, installer) 482 if dist is None: --> 483 raise DistributionNotFound(req) # XXX put more info here 484 to_activate.append(dist) 485 if dist not in req: <class 'pkg_resources.DistributionNotFound'>: scikits.pyaudiolab If I uncommend the line __version__ = require('scikits.pyaudiolab')[0].version, everything is fine.
If your code doesn't have any strange dependencies and you if email it to me (or point me to an url with a tgz), I could try and have a brief look at it, if you want.
You can find it here: http://www.ar.media.kyoto-u.ac.jp/members/david/archives/pyaudiolab-scikits.... (this is a tarball of the current source directory, as setup.py sdist does not work yet...). The only dependency is libsndfile, which is easily available if you use linux. David
David Cournapeau wrote:
I agree with you here, but that's a consequence of the history of numpy, I guess. Now, I don't find setuptools that much clearer myself. Maybe it is just because I have not used it extensively, but I always found setuptools (and distutils for that matter) quite complicated for anything not trivial.
I had to spend 2 hours to convert my project to setuptools, and for the only benefit to have support for namespace. And it still does not work because for some reasons, files automatically included by setuptools for install target are not included when using sdist target, whereas there is no difference between those targets with distutils... Well, I spend one hour trying to make something which is actually a bug, and is not even solved yet. Yeah...
http://www.mail-archive.com/distutils-sig@python.org/msg02794.html David
David Cournapeau wrote:
I got the following error when importing the package (from scikits import pyaudiolab)
<class 'pkg_resources.DistributionNotFound'>: scikits.pyaudiolab
If I uncommend the line __version__ = require('scikits.pyaudiolab')[0].version, everything is fine.
Change DISTNAME = 'pyaudiolab' to DISTNAME = 'scikits.pyaudiolab' -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
I tried the last tar file you gave, with the above correction but setup.pytest does not function on FC5 because of : pkg_resources.DistributionNotFound: numpy in the running build_ext. I tried that because I was trying to create a scikit on my own, but the same error occurs if I want "install_requires='numpy'" What are the needed versions of the differents libraries to create a scikit ? Matthieu 2007/5/22, David Cournapeau <david@ar.media.kyoto-u.ac.jp>:
Robert Kern wrote:
Change
DISTNAME = 'pyaudiolab'
to
DISTNAME = 'scikits.pyaudiolab
That was it, thanks,
David _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes:
I was just saying that if the only drawback of having a "fake" package is that it is not updated, it should not be too difficult to have something in place to test it automatically.
Yes, but I'd personally attach far higher priority to fixing mlabwrap if a test reveleaed a setup.py problem than fixing the scikits example package. However, I do agree that such a package would be useful provided there is someone or a group of people who'll take care of it, so if you feel like writing it, and don't think maintenance will be a problem, by all means do -- and if you like we can also replace all the mlabwrap examples on the wiki with ``scikits.example`` (or whatever you want to call it).
That's unfortunate -- I find numpy.distutils pretty impenetrable, and I also tend to believe that pretty much anything that's useful about it ought to eventually end up in setuptools, unless it's really specific to numpy or numerical python code -- IMO numpy having so much general-purpose building (and testing) facilities in numpy is a bad thing and increases the barrier of entry for scikits writers and other contributors of numerical python code unnecessarily, because the learning curve gets steeper and the number of wrong or largely equivalent choices and combinations that won't work grows. I agree with you here, but that's a consequence of the history of numpy, I guess.
Indeed and I'm sure that writing numpy.distutils was a good idea: distutils has been languishing for ages without an active maintainer (IIRC) and numpy/scipy clearly needed reasonable (and for some things even specialized) build support. I just think that with the arrival of setuptools (which is itself quite rich), it would be good to consolidate the two as much as possible and move to a situation where numpy.distutils really only deals with what can't and shouldn't be done by setuptools.
Now, I don't find setuptools that much clearer myself. Maybe it is just because I have not used it extensively, but I always found setuptools (and distutils for that matter) quite complicated for anything not trivial.
Yes, setuptools alone is already is enough to chew on. On the other hand setuptools really does offer extremely valuable functionality, first and foremost CPAN like functionality (i.e. easy automatic package installation for users) but also many really useful conveniences (such as the ``python setup.py develop``; ``test`` also seems like a good idea) and it is officially blessed for inclusion as distutils replacement -- so there's no way around dealing with it anyway and it will be with us for some time. So there seems more benefit in learning about setuptools than numpy.distutils, especially for functionality that could also be used for non-numpy projects (adding a dependency to numpy just so you can use numpy.distutils seems a bit excessive, right?), which is why I tried to mostly stick to setuptools on the wiki.
I had to spend 2 hours to convert my project to setuptools, and for the only benefit to have support for namespace.
I can comisserate -- all the work I've done on mlabwrap off late falls into the no-real-coding category, too. However, I are you sure that namespace support is the only benefit? Surely its a benefit if your users can say 'easy_install pyaudiolab' and it will just be downloaded and installed, automatically also downloading numpy if necessary?
And it still does not work because for some reasons, files automatically included by setuptools for install target are not included when using sdist target, whereas there is no difference between those targets with distutils...
Hmm maybe a bug? I recommend asking on the distutils-list, I 've found it quite helpful.
And I have not tested it yet on other platforms (I am always afraid of windows :) )
Rightly so, as I can attest from windows-user reports from the mlabwrap-user list.
I don't know much about numpy.distutils, but if you take a quick look at numpy/distutils, it is around 1.9 Mb, of which: - 1 M of tests - 350 ko for fortran (fcompiler) - 300 ko for extending distutils (command) - half of the rest seems pretty specific to numpy too (system_info is, cpuinfo too, other I don't know).
You could say that the modules of numpy/scipy fall into 3 categories wrt distutils: - pure python packages: not difficult to handle with distutils - python packages which use their own C extension: not difficult to handle with distutils.
From what I hear setuptools makes good cross-platform support a whole lot easier, but I can't comment on it from personal experience.
- python packages which depend on C/Fortran code (other libraries): difficult to handle with distutils, if not impossible.
Right -- I wonder whether it might sense to include support for that in setuptools, but it might be too specialized and better handled by numpy.distutils.
In my case, I use numpy.distutils to find a library and its header files. As far as I know, neither distutils or setuptools offer any facility for that.
Sounds like a good reason for using it then -- but wouldn't you agree that it would be even nicer if setuptools would also offer this functionality and you could use it for non-numpy-related projects? (not that I'd necessarily know if setuptools did already offer something like this)
What happens? I got the following error when importing the package (from scikits import pyaudiolab) [snipped]
I assume Robert Kern's tip solved the problem? cheers, 'as
Alexander Schmolck wrote:
So there seems more benefit in learning about setuptools than numpy.distutils, especially for functionality that could also be used for non-numpy projects (adding a dependency to numpy just so you can use numpy.distutils seems a bit excessive, right?), which is why I tried to mostly stick to setuptools on the wiki.
If you do not use numpy, then yes, it seems rather stupid to depend on numpy.distutils.
However, I are you sure that namespace support is the only benefit? Surely its a benefit if your users can say 'easy_install pyaudiolab' and it will just be downloaded and installed, automatically also downloading numpy if necessary?
I haven't tried it, but I am pretty sure it won't work because it depends on an external library anyway.
Hmm maybe a bug? I recommend asking on the distutils-list, I 've found it quite helpful.
It is a bug, but it seems it won't be resolved soon. So basically, I am doing everything in the configuration function provided by numpy.distutils for now. Anyway, I will add some info on the wiki, cheers, David
Alexander Schmolck wrote:
Hi everyone,
As the author of the first (if yet unreleased) scikit, I've tried to put some info for prospective scikit authors on
<http://projects.scipy.org/scipy/scikits/wiki/WikiStart>
that tries to condense down the most important points for those who are not yet familiar with the required infrastructure like setuptools (which is fairly rich) etc. to make things a little less overwhelming. A few things also still need clarification. As I'm myself just learning to get to grips with various of these things, so this page could use some attention from someone more knowledgable than me.
Here are a few specific points (search for FIXME/XXX in the document)
0. Blurp: What sort of things are suitable for a scikit project, what requirements must be met by projects or authors to qualify (this should also include something on acceptable/desirable[1] licenses, dependencies, code quality etc.) and what the overall aims and immediate future for scikits look like. I'm not really the person to write that so I just wrote something brief and vague.
1. Trac: I'm not really that familiar with trac, but I assume some sorts of conventions/infrastructure will need to be established in order for logically independent projects to use the same trac effectively. I guess one first step might be to use ``trac-admin`` to create each scikit (i.e. mlabwrap) as a component?
2. Homepage/Wiki/Docstandards etc. info.
Hmm, it occurs to me that a link to coding standards might also be a good idea...
3. Versioning: I think it would be nice if there were a standard way how people version their scikits, both conventions-wise as well as how the process is automated as far as is possible. I've outlined a setuptools-based solution in preference to one using numpy.distutils docs pretty opaque[1]. I found one problem with the approach you mention for versioning: when building a source package from sdist, only the release version part is appended to the source file, whereas with the "scipy approach", if your version is "0.7dev", it is appended to the source package. Also, having setuptools handling the versioning means it is more difficult to get the exact version back from other programs (things shell scripts and Makefile, for example), whereas extracting a string from a python file is trivial.
I am not sure to understand the advantages of the approach you are proposing ? David
David Cournapeau <david@ar.media.kyoto-u.ac.jp> writes:
I found one problem with the approach you mention for versioning: when building a source package from sdist, only the release version part is appended to the source file, whereas with the "scipy approach", if your version is "0.7dev", it is appended to the source package.
Well, I certainly did mean to have the 0.7dev bit appended to the source package, too and I thought the setup.cfg entry should take care of that, but apparently it doesn't.
Also, having setuptools handling the versioning means it is more difficult to get the exact version back from other programs (things shell scripts and Makefile, for example), whereas extracting a string from a python file is trivial.
I am not sure to understand the advantages of the approach you are proposing ?
None, since it obviously doesn't work properly. In an ideal world something like this should be triviually handled by the VC system. In a slightly less ideal world, the standard build tool (setuptools) should take care of it -- sane version handling certainly doesn't seem to have anything to do with numpy, but if that's the easiest way to do it for scikits and you know how to, by all means let's do that and update the wiki-page accordingly. Doing it with numpy.distutils actually was my first attempt but it didn't seem all that simple and easy either, and since I thought it didn't really belong there anyway, I decided to ask on the distutils-list, where I eventually got the setuptool-specific info that's currently on the wiki: <http://www.mail-archive.com/distutils-sig@python.org/msg03176.html> I think I've more than exhausted my time budget for this now; if needs be I'll just hardcode the version numbers for mlabwrap or write a 3 line-script that does it for me, but it would obviously great if you or someone else could come up with a better way and document it on the wiki. If you'd like to use the numpy.distutils approach, in the older wiki-versions and in the mlabwrap svn there's still an attempt at a fairly generic version.py file that I thought could get used for all scikits: <http://projects.scipy.org/scipy/scikits/browser/trunk/mlabwrap/version.py?rev=80> (NB. I think the file's location is wrong; version.py really ought to go under trunk/mlabwrap/scikits/mlabwrap) The corresponding setup.py, which tries to use numpy.distutils.Configuration just for the __svn_version__ file creation: <http://projects.scipy.org/scipy/scikits/browser/trunk/mlabwrap/setup.py?rev=88> Otherwise, it would presumably be best if someone asked on the distutils-list again for clarification. cheers, 'as
Alexander Schmolck wrote:
In an ideal world something like this should be triviually handled by the VC system. In a slightly less ideal world, the standard build tool (setuptools) should take care of it -- sane version handling certainly doesn't seem to have anything to do with numpy, but if that's the easiest way to do it for scikits and you know how to, by all means let's do that and update the wiki-page accordingly.
Doing it with numpy.distutils actually was my first attempt but it didn't seem all that simple and easy either, and since I thought it didn't really belong there anyway, I decided to ask on the distutils-list, where I eventually got the setuptool-specific info that's currently on the wiki:
<http://www.mail-archive.com/distutils-sig@python.org/msg03176.html>
I think I've more than exhausted my time budget for this now; if needs be I'll just hardcode the version numbers for mlabwrap or write a 3 line-script that does it for me, but it would obviously great if you or someone else could come up with a better way and document it on the wiki.
I don't have any solution for something automated, but I found the "scipy way" to work enough for me. The version is encoded in the info.py, the setup.py retrieve the version from there, and it is easily retrieved by a shell script. It is only hardcoded in one place. As I use a SCM different than svn, I cannot use the svn versioning provided by numpy.distutils. David
participants (8)
-
Alexander Schmolck -
Brian Hawthorne -
David Cournapeau -
dmitrey -
Jarrod Millman -
Matthieu Brucher -
Robert Kern -
Stefan van der Walt