[Distutils] EasyInstall: verbosity

[Ian Bicking]

Phillip J. Eby wrote:
> Yeah, this is actually exactly what I plan to do in an unspecified 
> future release, probably after I add filesystem 
> virtualization/sandboxing, or maybe in conjunction with it.  I hadn't 
> really thought about the part where main() would set it up, but that 
> part makes good sense too.  I was planning to pass verbosity options 
> through to the setup() script, and temporarily redirect its 
> stdout/stderr to pass through the logger.

I still haven't seriously used the logging module, despite its 
widespread applicability to things I do.  Go figure.  But it seems like 
this would be a very good application of it, except for the actual exec 
of the setup file, which would have to use capturing of stdout and 
stderr.  I'm guessing that redirection should be done to a sublogger, so 
that a caller can apply different filters and capturing to it.

> The idea there is that GUI tools that wrap easy_install may want to 
> display the progress messages, etc. in a progress dialog or status bar 
> or something of that sort.  This is especially true for the downloading 
> part, which can take a while.

Progress wouldn't fit in very well to logging, though.  I'm guessing 
there would have to be some notification object, with an API like:

     class INotifier:
         def describe_current_job(message):
             """Describe the currently-performed job, in the form of a
             short one-line text description"""
         def set_progress(progress, total, eta=None, units=None):
             """Indicate progress of the current job; progress and total
             are integers or floats.  eta is the estimated time to
             finish, in seconds (if not explicitly given the INotifier
             instance may guess).  units is an optional unit for progress
             and total (e.g., 'Mb', 'files')"""
         def reset_progress():
             """Reset any progress and estimated times"""

At least, that's my initial thought.  Potentially you could have two 
progress indicators, one over-all progress, and another for the current job.

> By the way, the sandboxing feature is where I plan to replace a few key 
> 'os' module functions and builtins with ones that "notice" if the setup 
> script is trying to modify files outside the installer's temporary 
> directory.  Initially this will just be so I can analyze those scripts' 
> behavior, but it might eventually grow into a facility to make them 
> think they're modifying files in the real filesystem, but they would 
> actually be getting redirected to the right temporary subdirectory for 
> stuff to be added into the egg.  I don't know if this feature is really 
> achievable, but most all file access in Python boils down to either 
> 'open()' or a call to an 'os' function imported from the 
> platform-specific extension ('nt', 'mac', 'posix', etc.).
> 
> Anyway, arguably people should "fix" their packages, but in practice 
> some folks won't, so a working sandbox could be helpful.  But I'll 
> probably want the sandbox tools to do a lot of output at various levels 
> of detail, hence why I'd probably use the logging package to do this.

Yikes, that sounds complicated.  Though sandboxing would be kind of neat 
in general, e.g., for mock filesystems in unit tests.  Are there 
particular packages you have in mind that need this?

>> I could add this, but the number of parallel patches is probably going
>> to leave everyone (including me) confused.  (I'm working on installation
>> of packages without setup.py files right now)
> 
> 
> Cool.  You're really taking the ball and running with it, which is what 
> I hoped would happen, as it's the specific reason I felt I had to 
> release an initial, usable version of EasyInstall this very weekend.  :)

Yes, I think The Time Is Right -- this sort of thing is really important 
for projects that integrate lots of libraries, and I think we're seeing 
that need from a few different directions.

The code seems very simple too, which is a good sign that it's solving 
the right problem.

-- 
Ian Bicking  /  ianb at colorstudy.com  / http://blog.ianbicking.org