Over on the matplotlib mailing list, we ran into a problem with trying
to use Universal newlines with gzip. In virtually all of my code that
reads text files, I use the 'U' flag to open files, it really helps not
having to deal with newline issues. Yes, they are fewer now that the
Macintosh uses \n, but they can still be a pain.
Anyway, we added such support to some matplotlib methods, and found that
gzip file reading broken We were passing the flags though into either
file() or gzip.open(), and passing 'U' into gzip.open() turns out to be
1) It would be nice if the gzip module (and the zip lib module)
supported Universal newlines -- you could read a compressed text file
with "wrong" newlines, and have them handled properly. However, that may
be hard to do, so at least:
2) Passing a 'U' flag in to gzip.open shouldn't break it.
I took a look at the Python SVN (2.5.4 and 2.6.1) for the gzip lib. I
# guarantee the file is opened in binary mode on platforms
# that care about that sort of thing
if mode and 'b' not in mode:
mode += 'b'
if fileobj is None:
fileobj = self.myfileobj = __builtin__.open(filename, mode
this is going to break for 'U' == you'll get 'rUb'. I tested
file(filename, 'rUb'), and it looks like it does universal newline
* Either gzip should be a bit smarter, and remove the 'U' flag (that's
what we did in the MPL code), or force 'rb' or 'wb'.
* Or: file opening should be a bit smarter -- what does 'rUb' mean? a
file can't be both Binary and Universal Text. Should it raise an
exception? Somehow I think it would be better to ignore the 'U', but
maybe that's only because of the issue I happen to be looking at now.
That later seems a better idea -- this issue could certainly come up in
other places than the gzip module, but maybe it would break a bunch of
code -- who knows?
I haven't touched py3 yet, so I have not idea if this issue is different
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
> [ Potential new "functools.partial_right", e.g.,
> split_comma = partial_right(str.split, '.')
Thanks for the feedback. Apologies if (as was suggested) this should
have gone to python-ideas; I thought as a fairly small extension to
existing functionality it would be OK here. I'll try to summarise the
There was some very luke-warm support.
Terry Reedy suggested it would be worth posting a patch to the tracker,
for the record, even if it turns out to be rejected.
Nick Coghlan made more clear than I did the main reason a
'partial_right' would be useful:
> [...] some functions and methods written in C (such as string methods)
> *don't* [support keyword args], so partial's keyword support doesn't
> A functools.rpartial would go some way towards addressing that.
On the other hand, Collin Winter asked for more evidence that real
benefit (beyond mere 'completeness' of the functools module) would
result. I don't really have to hand anything more than the three cases
mentioned in my original email (str.split, math.log, itertools.islice),
but since the change is so small, I thought the feature worth raising.
Leif Walsh pointed out that you could achieve the same effect by
defining your own function. This is true, but functools.partial exists
because it's sometimes useful to create such functions either more
concisely, or anonymously. A 'partial_right' would allow more such
functions to be so created.
Peter Harris was negative on the idea, pointing out that after
g = partial_right(f, 7)
you don't know which argument of 'f' the '7' is going to end up as,
because it depends on how many are supplied in the eventual call to 'g'.
This is true, and would require some care in partial_right's use. Peter
> There's probably a reason why Haskell doesn't do this...
I have only written about five lines of Haskell in my life, so take this
with a hefty pinch of salt, but: Haskell does have a 'flip' function
which reverses the order of a function's arguments, so it looks like you
can very easily build a 'partial_right' in Haskell, especially since
standard functions are in curried form.
There was some discussion (started by Antoine Pitrou) of an idea to
generalise 'partial' further, potentially using the Ellipsis object, to
allow arbitrarily-placed 'holes' in the argument list. E.g.,
split_comma = partial(str.split, ..., ',')
In some ways I quite like the even-more-completeness of this idea, but
think that it might be the wrong side of the complexity/benefit
trade-off. Scott David Daniels pointed out that using Ellipsis would
have the downside of
> [...] preventing any use of partial when an argument could be an the
> Ellipsis instance.
This could be fixed by making the general form be something with the
partial_explicit(f, hole_sentinel, *args, **kwargs)
where appearances of the exact object 'hole_sentinel' in 'args' would
indicate a hole, to be filled in at the time of the future call. A user
wanting to have '...' passed in as a true argument could then do
g = partial_explicit(f, None, 3, ..., 4, axis = 2)
hole = object()
g = partial_explicit(f, hole, 3, ..., hole, 4, axis = 2)
if they wanted a true '...' argument and a hole. (I might have the
syntax for this wrong, having not played with Python 3.0, but I hope the
idea is clear.)
There was some concern expressed (by Daniel Stutzbach, Alexander
Belopolsky) that the meaning of '...' would be confusing --- 'one hole'
or 'arbitrary many holes'?
I think the extra complexity vs extra functionality trade-off is worth
considering for 'partial_right', but my personal opinion is that a
'partial_explicit' has that trade-off the wrong way.
I'll try to find time to create the patch in the tracker in the next few
days, by which time perhaps it'll have become clearer whether the idea
is a good one or not.
> On Thu, Jan 29, 2009 at 6:12 AM, Ben North <ben(a)redfrontdoor.org> wrote:
>> I find 'functools.partial' useful, but occasionally I'm unable to use it
>> because it lacks a 'from the right' version.
For me, the main objection to a partial that places
its stored positional arguments from the right is
that you don't know which positions those arguments
will actually occupy until the partial is called.
Who *really* thinks that would be a neat feature? There's probably a
reason why Haskell doesn't do
> Paul Moore <p.f.moore(a)gmail.com> wrote:
> Serious question: does anybody know how to get better communication
> from the user base? My impression is that it's pretty hard to find out
> who is actually using 3.0, and get any feedback from them. I suppose a
> general query on clp might get some feedback, but otherwise, what?
> I've not seen any significant amount of blog activity on 3.0.
I teach programming in a CS dept. at a UK university. We've
been teaching Python in one context or another for 5 years now,
and are currently in our second year of teaching it as the
primary programming language.
We have to make decisions on software versions for the coming
academic year during the summer months. This means that we've
had to be content this year with Python 2.5.
We'd love to switch to 3.0 as soon as possible (i.e., Oct 2009),
as it is a significantly cleaner language for our purposes.
However, we make extensive use of third-party libraries and
frameworks such as Pygame, wxPython, etc, to increase the
motivation levels of students. The 3.0-readiness of these
libraries and frameworks is inevitably going to be a factor in
the decision we make this summer.
I was thinking of adding a breakpoint opcode to python to enable less
I came across posts from 1999 by Vladimir Marangozov and Christian Tismer
discussing this issue but the links to the code are all out of date.
Did anything come of this? Is this a good approach to take? - if so why was
this never incorporated?
Dr. Andrew Perella
Chief Software Architect
Eutechnyx Limited. Metro Centre East Business Park, Waterside Drive,
Gateshead, Tyne & Wear NE11 9HU UK Co.Reg.No. 2172322
T +44 (0) 191 460 6060 F +44 (0) 191 460 2266 E
<mailto:firstname.lastname@example.org> ajp(a)eutechnyx.com W
This e-mail is confidential and may be privileged. It may be read, copied
and used only by the intended recipient. No communication sent by e-mail to
or from Eutechnyx is intended to give rise to contractual or other legal
liability, apart from liability which cannot be excluded under English law.
This e-mail is confidential and may be privileged. It may be read, copied and used only by the intended recipient. No communication sent by e-mail to or from Eutechnyx is intended to give rise to contractual or other legal liability, apart from liability which cannot be excluded under English law.
This email has been scanned for all known viruses by the Email Protection Agency. http://www.epagency.netwww.eutechnyx.com Eutechnyx Limited. Registered in England No: 2172322
In the past couple of days, test_distutils started failing.
It looks like a pure python error and may have been
introduced by guilherme.polo's checkins:
File "c:\py27\lib\distutils\tests\test_sdist.py", line 119, in test_make_distr
File "c:\py27\lib\distutils\spawn.py", line 37, in spawn
_spawn_nt(cmd, search_path, dry_run=dry_run)
File "c:\py27\lib\distutils\spawn.py", line 70, in _spawn_nt
cmd = _nt_quote_args(cmd)
File "c:\py27\lib\distutils\spawn.py", line 61, in _nt_quote_args
args[i] = '"%s"' % args[i]
TypeError: 'str' object does not support item assignment
1 test failed:
I'm currently working on a patch that adds the __thiscall calling
convention to ctypes. This calling convention is used on Windows
for calling member functions from C++ classes. The idea is to eventually
allow ctypes to wrap C++ classes.
To test this functionality it is required to add some C++ source code to the
ctypes private test module _ctypes_test.pyd/_ctypes_test.so.
Is it appropriate to add C++ source files to the Python repository,
or would that create too much trouble on some platforms?
In the Python Argentina mail list there's already people passing
examples and asking help about Python 3.
This introduces the problem that some examples are in Py2 and others
are in Py3. Sometimes this is not explicit, and gets confusing. I'm
trying to avoid this confusion when preparing my own examples. So far,
I use (py3) as a prefix for any example block, like:
>>> (some example)
Is there any recommended way to avoid confusion in these cases? (I'm
thinking about changing the prompt in my Python installation, to
something like ">2>>" and ">3>>", to be explicit about it... but I
wanted to know if there's another better way)
I'd like to write a small daemon in Python, which should never be
swapped out (on Linux, this daemon will be Linux specific, so no need
in a platform-independent solution).
In C I'd do:
//do stuff here
Is there anything similar in Python?
TIA & kind regards