"Filip M. Noetzel" <filip(a)j03.de> schrieb:
> (I'm replying out of band, [...]
I hope you don't mind if I send a copy of my reply back to the list,
though.
> I think wrote what you are describing in your post a few months ago:
>
> http://pypi.python.org/pypi/buildout-md5sums (Source at https://github.com/peritus/buildout-md5sums )
It has a very similar purpose indeed. Nice to see that this is
something I'm not the only one to want to have. Thank you for pointing
it out!
> I'd love feedback on it (I use it on a day-to-day basis for my buildouts, but don't know other users).
The problems I see with your approach:
- Patching the download API is technically less than optimal.
- Anchoring MD5 enforcement that deeply within the mechanics means that
client code cannot decide whether its associated configuration needs
to honour the allow-picked-downloads flag. I'm not sure whether
that's a good thing or bad - that's part of what I'm hoping to
discuss. I could imagine that one wants to enforce checksums for,
e.g., source packages downloaded by a cmmi recipe while avoiding them
for base configuration files downloaded by buildout itself.
- As a less technical aspect, you might want to consider a more serious
licence for your package if you hope for more wide-spread use.
Thomas
On Sun, Mar 6, 2011 at 20:49, cool-RR <cool-rr(a)cool-rr.com> wrote:
> Speaking of names, I would rename PyPI to packages.python.org, maybe move
> the existing documentation center to docs.python.org, and then move the docs
> of Python itself to a `/python` folder...
> But that's just me.
No, I do think docs.python.org should be for documentation of Python.
I agree having package documentation on packages.python.org may not be
very pedagogical, though. :-)
//Lennart
At 08:08 AM 3/10/2011 -0500, Jim Fulton wrote:
>I really should let this rest ....... really ....
I notice you seem to have already let rest defending your proposal,
as opposed to opposing mine. ;-)
That is, I don't see where you've included any counterargument for
why convincing people to *stop* using "package" for "python package"
isn't going to be a lot harder than simply *adding* the term
"project" to existing docs, via simple explanations like this one:
"If you would like to distribute your python package or module, you
need to create a project. A project consists of a directory
containing the package(s) or module(s) you want to distribute, along
with a setup.py/.cfg, specifying the project's name, current version
number, and other information. If you wish to distribute your
package via PyPI, then you must choose a unique project name and
register it. If you are only distributing one module or package,
you can name the project after it, as long as it doesn't conflict
with any other project names already registered on PyPI."
Context-specific explanations like this are easy to give, at a place
where the recipient of the explanation *wants* to learn something.
In contrast, to convince everybody in the world to replace "package"
with "module", you must send out a constant broadcast to people who
don't really care.
(your other points are addressed below)
>On Wed, Mar 9, 2011 at 11:43 AM, P.J. Eby <pje(a)telecommunity.com> wrote:
> > At 07:06 AM 3/9/2011 -0500, Jim Fulton wrote:
> >>
> >> They certainly aren't "projects" in any sense that most people would
> >> understand.
> >
> > I don't follow you. Sourceforge hosts projects. Freshmeat indexes
> > projects. Mozdev.org hosts projects. The Apache Foundation hosts projects.
> > "Project", IOW, is *exactly* the word people use for these things, in the
> > larger world of software, and that's precisely why I chose it for my
> > original terminology proposal.
>
>But the things in PyPI are not projects. Projects have bug trackers,
>and mailing lists, and source code repositories.
And which of those things have to be hosted on the same site as the
project's index page? Freshmeat, for example, is not a project
*hosting* system, it's a project *index*. PyPI is in that sense, a
project index. Some index pages may link to all the stuff associated
with that project, some may not.
(i.e. the set of all possible project features is not equal to the
intersection of all projects' actual features: some project have
mailing lists and not bug trackers, and vice versa. This does not
make them non-projects.)
> Projects are organizations of people -- not software distributions.
I'm an organization of people; am I a project? No, the various
things I have listed on PyPI are my projects, and I distribute
releases of them.
>The things in PyPI that you call "projects" are things that get
>produced by projects. There isn't a buildout.recipe.egg
>"project". buildout.recipe.egg is just a um packaging of some software
>components. It's not a "project" in any usual sense of the word.
I don't get that. You work on it, and it is distinct from other
works. That makes it a project.
>No. End users need to know about this. They need to know what these
>names mean and that the "package name" (distutils) or "project name"
>(setuptools) doesn't imply that the distribution will provide a python
>package of the same name. Trust me on this. Users care about this.
Actually, I believe distutils calls it a "distribution name", but I
could be wrong about that. ;-)
> > For one thing, if the distutils documentation simply starts out like:
> >
> > "If you want to distribute your work to the larger Python community, you
> > need to create a project for it. In practical terms, this means having a
> > directory with a setup.py and the code, data, or documentation you wish to
> > distribute. Your project will also need a unique name, if you want to make
> > it accessible to others via the Python Project Index (PyPI)." (replace
> > setup.py w/setup.cfg for distutils2, of course)
>
>It does? Where? Can you give a link? I don't see this text anywhere.
Of course not; it was a proposal for a hypothetical
introduction. That's why it says "if" up there. ;-)
> > This usage of project also maps onto common IDE usage of the term
> project --
> > a directory of stuff that you're going to edit and build.
>
>That use of project is equivalent to "working directory" and not
>relevant, IMO.
Huh? In an IDE, a project is a thing you build and distribute,
usually with source control. It's distinct from other projects, and
it has a name. It is usually not tied to a specific code unit such
as a module or package (regardless of what the code units of the
relevant programming language are called). It has additional
metadata, over and above the actual code units, and possibly includes
non-code resources such as documentation, data, images, etc.
IOW, it's *exactly* the sort of project we're talking about
here. So, the term project is consistent with both the IDE usage of
the word, *and* the mozdev/freshmeat/sourceforge et al meaning of project.
It is thus unlikely to be confusing or surprising in the least.
(Side question: were you *really* confused by the term project when
you first began working with setuptools? I don't remember you having
any issues with it at the time, but perhaps I've forgotten?)
The hard part of introducing the term *here*, on this mailing list,
is precisely the same part that is hard for your proposal: it is
usually happening at a point where people are *not* as open to
learning something.
When somebody is on the list and we need to explain the
project/package distinction, it's noise to them because they're
trying to solve some other problem, and the distinction doesn't seem important.
At the point, however, where someone is reading the distutils docs
for the first time, or reading a "how to distribute your stuff"
tutorial, that is precisely the place where that distinction will be
perceived as valuable.
That is, they are asking, "what do I need to do to get my code out
there?" And the answer is, "make a project, register it, and upload
some files." This answer is sensible and satisfying, so the term
slips right in.
Arguably, you could call it a "fooblymuffin" at that same point of
introduction, and the "it's an answer to their question" argument
mostly still applies. But the pre-existing meaning of "project" from
both IDE usage and project-sites usage means that anyone who's
encountered the term in either of those uses will likely have an
immediate "aha" or "of course" reaction, thereby making the learning
and acceptance even easier.
Renaming package to module, on the other hand, has no similarly
obvious point of motivation to learn, except for people who are *not
yet python developers*... which means that it's going to take a
long, long time to get the terminology changed.
At 07:06 AM 3/9/2011 -0500, Jim Fulton wrote:
>They certainly aren't "projects" in any sense that most people would
>understand.
I don't follow you. Sourceforge hosts projects. Freshmeat indexes
projects. Mozdev.org hosts projects. The Apache Foundation hosts
projects. "Project", IOW, is *exactly* the word people use for these
things, in the larger world of software, and that's precisely why I
chose it for my original terminology proposal.
Many PyPI listings also only describe a project, in the sense that
there is no release information at all, or the code is only available
via a revision control system, etc.
>The term "project" has has never stuck, despite being discussed
>repeatedly.
It's only been discussed here, on this list. It hasn't been put in
official documentation, or really blessed by anybody. When it has
been discussed, it's been considered a reasonable name, and when
people have needed to make the distinction, they've tended to use it.
The primary reason it hasn't caught on in a larger sense is that
people don't know about it, and have no motivation to learn it unless
they end up in a situation where the distinction makes a difference...
And let's face it, it really only makes a difference if you are
creating some kind of packaging or distribution tool, or if you have
a corner case of using one.
>I think given the historical use of the term "package" it
>was reasonable to find another term. OK, we tried. It didn't work.
I don't think that's an accurate assessment on two points. First,
"we" didn't try -- *I* did. (It's possible that someone else has
promoted it, but I don't recall any instances of that.)
So, there actually being a "we" behind it -- where "we" includes an
approved PEP, documentation, etc., would be helpful.
Second, I don't think it's accurate to say "it didn't work". It's
not like people who need the distinction have objected to the word or
proposed any alternatives.
>We can pretent that if we work hard enough, we can make people adopt
>our confusing terminology. Good luck with that.
No matter which way we go (assuming there aren't any other
alternatives), we will be trying to get some group of people to
change their terminology. I'm just pointing out that the group that
would need to change to use "project" is both smaller *and*
inherently more motivated to change their usage, than the group that
would need to replace "package" with "module".
Thus, if getting people to use "project" is an uphill battle, getting
people to stop using "package" is going to be a much higher vertical
cliff. ;-)
For one thing, if the distutils documentation simply starts out like:
"If you want to distribute your work to the larger Python community,
you need to create a project for it. In practical terms, this means
having a directory with a setup.py and the code, data, or
documentation you wish to distribute. Your project will also need a
unique name, if you want to make it accessible to others via the
Python Project Index (PyPI)." (replace setup.py w/setup.cfg for
distutils2, of course)
This usage of project also maps onto common IDE usage of the term
project -- a directory of stuff that you're going to edit and build.
And, it immediately introduces the term to a superset of the audience
that will need to know it. Having PyPI describe its contents as
"projects" is pretty much the other half of the indoctrination needed.
In contrast, to make the package->module change, you'd have to not
only change the official language reference and tutorial, but also
every third-party book or tutorial about Python.
Sure, some of those third party references would also exist for
"package" as "project", but that's simply an *imprecise* usage,
rather than an *incorrect* one.
At 05:18 PM 3/7/2011 -0500, Jim Fulton wrote:
>If what we now call "packages" were called "modules", then we could
>start using the term "package" the way everyone else does. I think
>lots of people would be less confused.
It seems to me that in order to make that change, you have to get
more people to change their terminology, since the set of people who
need to refer to package[module] is larger than the set of people who
need to refer to package[project]. (There is also a larger body of
documentation associated with package[module].)
IOW, I think this proposal is a heavy uphill battle, both in the
number of people to be convinced and the amount of documentation. In
addition, the people who are calling a project a package can more
easily understand the need to call it a project, than the people who
are calling a package a package, will understand the need to call it
a module. ;-)
>Otherwise, I prefer we try hard to use the precise definitions
>above. This topic can be confusing enough without making it more so
>through sloppy terminology.
I think that this approach is more achievable: it requires only an
official blessing, a relatively small amount of documentation to be
changed, and the renaming of PyPI et al. (i.e. "Python Projects
Index", projects.python.org, etc.)
("Projects Index" is a better name anyway, since some things on PyPI
are not packages at all but applications, scripts, modules, plugins, etc.)
At 11:46 AM 3/7/2011 +0100, Lennart Regebro wrote:
>I'll make a note of this in the documentation to clear it up.
>Distutils2 is definitely in the minority at the moment when it comes
>to calling them "projects".
The term has been in use in setuptools since around 2005, but it
hasn't caught on much outside of the small group of people who need
to be able to speak precisely about the concept. ;-)
If you search the sig archives, though, you will find that it gets
proposed and mostly-approved every time the topic comes up, of what
to call these "things we distribute releases of". The more-or-less
consensus terminology (for people who need a precise terminology):
package = thing you import in Python that contains modules
project = thing you make releases of
release = one version of a project
distribution = a file that embodies the release of a project (may be
source or binary)
People who don't care about precision just call everything a package,
pretty much. Heck, a lot of times I find myself starting to type
"package" when I'm really talking about a project, release, or
distribution, despite promoted the more-precise terminology for half
a decade or so. ;-)
Hi,
I wonder whether it makes sense to introduce a new option to the buildout
section, named downloads-require-md5sum or similar. Recipes such as
zc.recipe.cmmi or gocept.download could then be made to require an MD5
checksum for every downloaded resource if that option is set to true, and
treat checksums as optional otherwise.
The responsibility for evaluating that option would have to lie with
recipes, not buildout's download API for two reasons: the download API
should stay as simple as possible, and it must remain possible to use it
for downloading resources such as base configurations (by the extends
option) whose MD5 checksum isn't known beforehand.
Any opinions on whether such functionality would be worth the added weight
of such an option, or maybe insights into why requiring checksums
depending on a buildout option might not be useful after all? Thank you.
--
Thomas
On Mon, Mar 7, 2011 at 21:47, P.J. Eby <pje(a)telecommunity.com> wrote:
> The term has been in use in setuptools since around 2005, but it hasn't
> caught on much outside of the small group of people who need to be able to
> speak precisely about the concept. ;-)
Well, I've changed the terminology in pyroma, to hopefully spread it
around a bit.
//Lennart
I've started working on a little utility to give a quality rating on
packages, expressed in 0-10 points, and also in cheese types,
according to smellyness.
It's going to check for things like that it has all meta data it
should have, such as author_email, specifies Python versions via the
trove classifiers (currently works) and that it specifies all
dependencies (still todo). It will support both checking on a package
(works currently) a distribution file and PyPI (still to do).
It's not a uniqe idea, it overlaps with Andreas Jungs
zopyx.trashfinder in scope, and it will also in the case of checking a
package on PyPI check that there are several people that have owner
access, and hence include the functionality of mr.parker. (In fact
when checking on PyPI it will also check if there are documentation on
packages.python.org, that the distribution files are uploaded to PyPI,
etc, but this is all still todo). But I didn't find anything else, and
I wanted bigger scopes than both these in what to check in and which
cases.
But, before I move this to a public repository and upload it to PyPI,
there is one important thing to be determined: What should it be
called? Currently I'm calling it "pypilib.quality". I don't mind this
kind of boring names, but there is currently not a pypilib namespace,
and I don't want to just create top level namespaces left and right
for no reason. So other names are welcome. It doesn't have to have a
namespace either.
In the long run I would not mind to see this utility integrated into a
general pypi/cheeseshop script with other utility commands, which even
could include installing and removing, thusly giving Perl people what
they think they want a "CPAN" for Python. :-)
--
Lennart Regebro: http://regebro.wordpress.com/
The Python 3 Porting book is out: http://python3porting.com/
+33 661 58 14 64
An annoyance in buildout configurations, resulting from it's use of
ConfigParser, is that configuration data lines are stripped of leading
spaces. This means configurations like this:
[paste.ini]
recipe = zc.recipe.deployment:configuration
text =
[app:main]
use = egg:bobo
bobo_resources = cmsapp
filter-with = zodb
[filter:zodb]
use = egg:zc.zodbwsgi
configuration =
<zodb main>
<zeoclient>
server ${main-db-server:address}
</zeoclient>
</zodb>
<zodb index>
<zeoclient>
server ${index-db-server:address}
</zeoclient>
</zodb>
[server:main]
use = egg:Paste#http
host = localhost
port = 8080
Don't work because leading whitespace is stripped, forcing me to use
hacks like this::
[paste.ini]
recipe = zc.recipe.deployment:configuration
s =
text =
${:s}[app:main]
${:s}use = egg:bobo
${:s}bobo_resources = cmsapp
${:s}filter-with = zodb
${:s}
${:s}[filter:zodb]
${:s}use = egg:zc.zodbwsgi
${:s}configuration =
${:s} <zodb main>
${:s} <zeoclient>
${:s} server ${main-db-server:address}
${:s} </zeoclient>
${:s} </zodb>
${:s} <zodb index>
${:s} <zeoclient>
${:s} server ${index-db-server:address}
${:s} </zeoclient>
${:s} </zodb>
${:s}
${:s}[server:main]
${:s}use = egg:Paste#http
${:s}host = localhost
${:s}port = 8080
:(
I propose to give buildout it's own parser that begaves like
ConfigParser, with the exeption that leading whitespace won't be
stripped, but will be dedented with textwrap.dedent.
There is some potential for breakage as some option values without
leading whitespace before would get it now, but my sense is that this
would be unlikely.
Thoughts? If I did that (and I really want to do this) I'd release a
beta to help people assess the impact.
Jim
--
Jim Fulton
http://www.linkedin.com/in/jimfulton
