From zac256 at gmail.com  Sat Aug  1 20:28:05 2009
From: zac256 at gmail.com (Zac Burns)
Date: Sat, 1 Aug 2009 11:28:05 -0700
Subject: [Python-ideas] os.listdir with current working directory as
	default
In-Reply-To: <h0njl7$r39$1@ger.gmane.org>
References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com>
	<94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com>
	<gv7dh8$tbb$1@ger.gmane.org>
	<94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com>
	<91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com>
	<gv99p5$n7u$1@ger.gmane.org>
	<94bdd2610905231708o1222551axf7ef449fc1329621@mail.gmail.com>
	<loom.20090524T213840-1@post.gmane.org>
	<333edbe80906091713ma098b0bwd20491725c9cac71@mail.gmail.com>
	<h0njl7$r39$1@ger.gmane.org>
Message-ID: <333edbe80908011128l51897880x9d5adaf519ca63fe@mail.gmail.com>

Yes, I always do use the full path.

--
Zachary Burns
(407)590-4814
Aim - Zac256FL
Production Engineer (Digital Overlord)
Zindagi Games



On Tue, Jun 9, 2009 at 11:25 PM, Lie Ryan<lie.1296 at gmail.com> wrote:
> Zac Burns wrote:
>>> You'd better be careful with it. The "current directory" is process-wide and so
>>> modifying it in a thread will affect all other threads in your program. I'm not
>>> saying all programs have this characteristic, but relying on the current
>>> directory rather than explicit paths isn't always a good idea...
>>
>> I agree with Antoine, but would go further to say that making it a
>> default argument is an enablement - just making it easier for people
>> to follow a bad pattern.
>>
>> This doesn't apply to all default arguments of course, just that the
>> working directory is "considered harmful" and we probably shouldn't
>> make changes that make it easier for people to use.
>
> When you open() files do you always use full path? Why don't we
> eliminate all modules and built-in function that uses relative path by
> default then? That way we will always be safe? Making open(), etc raise
> RelativePathError if using relative path.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From digitalxero at gmail.com  Mon Aug  3 07:44:54 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Sun, 2 Aug 2009 23:44:54 -0600
Subject: [Python-ideas] etree improvements
Message-ID: <e9764b730908022244v7680cc05nc9f6777392c0ba9@mail.gmail.com>

Well I find myself converting a project from an OLD version of minidom
(from back 1999  or so) to etree. And I find myself  wishing there was
something like the Element(Tree).findall / Element(Tree).getiterator
pair for Element(Tree).find and Element(Tree).findtext, like a
Element(Tree).iter_find / Element(Tree).iter_findtext that would
recurse till the whole tree is walked or they found the tag / path you
specified.

Something like http://dpaste.com/hold/74590/


From sridharr at activestate.com  Tue Aug  4 18:11:42 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 04 Aug 2009 09:11:42 -0700
Subject: [Python-ideas] Meta idea - monthly digest of python-ideas
Message-ID: <op.ux5a9nur1q4jlt@whymac>

1. Create a blog
2. Create a summary of each thread
3. On every month, post the summaries in the blog

The summary of a thread may adhere to a standard template, for instance,  
definition of the idea, its merits, and criticisms.

-srid


From phd at phd.pp.ru  Tue Aug  4 18:25:14 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Tue, 4 Aug 2009 20:25:14 +0400
Subject: [Python-ideas] Meta idea - monthly digest of python-ideas
In-Reply-To: <op.ux5a9nur1q4jlt@whymac>
References: <op.ux5a9nur1q4jlt@whymac>
Message-ID: <20090804162514.GB24468@phd.pp.ru>

On Tue, Aug 04, 2009 at 09:11:42AM -0700, Sridhar Ratnakumar wrote:
> 2. Create a summary of each thread
> 3. On every month, post the summaries

   Wanna work on this?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From tlesher at gmail.com  Tue Aug  4 18:49:37 2009
From: tlesher at gmail.com (Tim Lesher)
Date: Tue, 4 Aug 2009 12:49:37 -0400
Subject: [Python-ideas] Meta idea - monthly digest of python-ideas
In-Reply-To: <op.ux5a9nur1q4jlt@whymac>
References: <op.ux5a9nur1q4jlt@whymac>
Message-ID: <9613db600908040949j67da04b0y524ba28786f6c1e5@mail.gmail.com>

On Tue, Aug 4, 2009 at 12:11, Sridhar
Ratnakumar<sridharr at activestate.com> wrote:
> 1. Create a blog
> 2. Create a summary of each thread
> 3. On every month, post the summaries in the blog

You mean like the Python-Dev summaries?

http://www.python.org/dev/summary/

-- 
Tim Lesher <tlesher at gmail.com>


From SridharR at activestate.com  Tue Aug  4 19:53:57 2009
From: SridharR at activestate.com (Sridhar Ratnakumar)
Date: Tue, 04 Aug 2009 10:53:57 -0700
Subject: [Python-ideas] Meta idea - monthly digest of python-ideas
In-Reply-To: <20090804162514.GB24468@phd.pp.ru>
References: <op.ux5a9nur1q4jlt@whymac> <20090804162514.GB24468@phd.pp.ru>
Message-ID: <op.ux5fz7s0brrvlq@double>

On Tue, 04 Aug 2009 09:25:14 -0700, Oleg Broytmann <phd at phd.pp.ru> wrote:

> On Tue, Aug 04, 2009 at 09:11:42AM -0700, Sridhar Ratnakumar wrote:
>> 2. Create a summary of each thread
>> 3. On every month, post the summaries
>   Wanna work on this?

No - which is why I posted this as an idea here. :-) Maybe someone will  
take that responsibility .. or not.

-srid


From fuzzyman at gmail.com  Tue Aug  4 19:58:29 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Tue, 4 Aug 2009 18:58:29 +0100
Subject: [Python-ideas] Meta idea - monthly digest of python-ideas
In-Reply-To: <op.ux5a9nur1q4jlt@whymac>
References: <op.ux5a9nur1q4jlt@whymac>
Message-ID: <6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com>

2009/8/4 Sridhar Ratnakumar <sridharr at activestate.com>

> 1. Create a blog
> 2. Create a summary of each thread
> 3. On every month, post the summaries in the blog
>
> The summary of a thread may adhere to a standard template, for instance,
> definition of the idea, its merits, and criticisms.


It's a *great* idea, but without someone (or preferably plural) to take it
on it won't go anywhere.

Michael


>
>
> -srid
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090804/97b170aa/attachment.html>

From kristjan at ccpgames.com  Tue Aug  4 23:39:37 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 4 Aug 2009 21:39:37 +0000
Subject: [Python-ideas] adding a "path" component to XMLRPC dispatch objects
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098493C5F3@exchis.ccp.ad.local>

I've created the following on Rietveld:
http://codereview.appspot.com/100046

Here's the descriptions:
by passing the "path" component of the xmlrpc request to the dispatch method, it
becomes possible to dispatch differently according to this.  This patch provides
that addition.  Additionally, it provides an MultiPathXMLRPCDispatcher mixin
class and a MultiPathXMLRPCServer that uses it, to have multiple dispatchers for
different paths.
This allows a single server port to serve different XMLRPC servers as
differentiated by the HTTP path.  A test is also preovided.

I intend this as an addition to the standard lib.  Any thoughts?

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090804/9de9d1f3/attachment.html>

From tleeuwenburg at gmail.com  Thu Aug  6 03:01:20 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 6 Aug 2009 11:01:20 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
Message-ID: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>

this could be nicer than prefixing things with is, and makes things clearer
in some cases, e.g.

def shouldSomethingHappenFlag(self, context):
   if 'relevant' in context:
     return True

   return False

vs

def shouldSomethingHappen?(self, context):
  if 'relevant' in context:
    return True
  return False

I just think it looks nicer, and is a good hint that a true/false or other
kind of flag is likely to come back.

Cheers,
-T

-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090806/50cde196/attachment.html>

From contact at xavierho.com  Thu Aug  6 09:50:34 2009
From: contact at xavierho.com (Xavier Ho)
Date: Thu, 6 Aug 2009 17:50:34 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
Message-ID: <2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com>

On Thu, Aug 6, 2009 at 11:01 AM, Tennessee Leeuwenburg <
tleeuwenburg at gmail.com> wrote:

> this could be nicer than prefixing things with is, and makes things clearer
> in some cases, e.g.
>
> def shouldSomethingHappenFlag(self, context):
>    if 'relevant' in context:
>      return True
>
>    return False
>
> vs
>
> def shouldSomethingHappen?(self, context):
>   if 'relevant' in context:
>     return True
>   return False
>
> I just think it looks nicer, and is a good hint that a true/false or other
> kind of flag is likely to come back.
>

What about

hasSomethingHappened(self, context)

or

isRelevant(self, context)

Either way.. I think ? has better uses though.

Regards,

Ching-Yun "Xavier" Ho, Technical Artist

Contact Information
Mobile: (+61) 04 3335 4748
Skype ID: SpaXe85
Email: contact at xavierho.com
Website: http://xavierho.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090806/7b73d51f/attachment.html>

From tleeuwenburg at gmail.com  Thu Aug  6 10:10:25 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 6 Aug 2009 18:10:25 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com>
Message-ID: <43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com>

On Thu, Aug 6, 2009 at 5:50 PM, Xavier Ho <contact at xavierho.com> wrote:

> On Thu, Aug 6, 2009 at 11:01 AM, Tennessee Leeuwenburg <
> tleeuwenburg at gmail.com> wrote:
>
>> this could be nicer than prefixing things with is, and makes things
>> clearer in some cases, e.g.
>>
>> def shouldSomethingHappenFlag(self, context):
>>    if 'relevant' in context:
>>      return True
>>
>>    return False
>>
>> vs
>>
>> def shouldSomethingHappen?(self, context):
>>   if 'relevant' in context:
>>     return True
>>   return False
>>
>> I just think it looks nicer, and is a good hint that a true/false or other
>> kind of flag is likely to come back.
>>
>
> What about
>
> hasSomethingHappened(self, context)
>
> or
>
> isRelevant(self, context)
>
> Either way.. I think ? has better uses though.
>

Well, probably everyone will want to do things differently. Not all method
names work with an 'is' stuck on the front, or a 'Flag' stuck on the end...
I'm trying out an idiom of ending True/False returners with 'Q', i.e.
self.isSomethingTheCaseQ() or self.shouldSomethingHappenQ() or
self.getSomeFlagQ()... I find that's reasonably aesthetically pleasing, and
is a good hint about what the method is for.

Also, when working with someone else's code, they may already has isFoo()
style methods which don't return flags.

So Q is good enough for me, but ? would be even better, for me.

Cheers,
-T
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090806/fdbd1946/attachment.html>

From contact at xavierho.com  Thu Aug  6 10:30:14 2009
From: contact at xavierho.com (Xavier Ho)
Date: Thu, 6 Aug 2009 18:30:14 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com>
	<43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com>
Message-ID: <2d56febf0908060130q375994ej87bf51f665e6bc27@mail.gmail.com>

On Thu, Aug 6, 2009 at 6:10 PM, Tennessee Leeuwenburg <
tleeuwenburg at gmail.com> wrote:

> Well, probably everyone will want to do things differently. Not all method
> names work with an 'is' stuck on the front, or a 'Flag' stuck on the end...
> I'm trying out an idiom of ending True/False returners with 'Q', i.e.
> self.isSomethingTheCaseQ() or self.shouldSomethingHappenQ() or
> self.getSomeFlagQ()... I find that's reasonably aesthetically pleasing, and
> is a good hint about what the method is for.
> Also, when working with someone else's code, they may already has isFoo()
> style methods which don't return flags.
>
> So Q is good enough for me, but ? would be even better, for me.
>
> Cheers,
> -T
>

It's cool, I'd like to know what others think about it. Although I don't
think many would agree with you, as this is more of a personal style thing.

Q looks good, and obvious. Or you could resort to the traditional style
where b_ prefix seemed to be loved by many C programmers. Hmm.

isXX and hasXXX still work for me well though.

Take it easy,
-Xav
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090806/f7694429/attachment.html>

From ben+python at benfinney.id.au  Thu Aug  6 11:40:51 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 06 Aug 2009 19:40:51 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com>
	<43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com>
Message-ID: <877hxhnuu4.fsf@benfinney.id.au>

Tennessee Leeuwenburg
<tleeuwenburg at gmail.com> writes:

> I'm trying out an idiom of ending True/False returners with 'Q', i.e.
> self.isSomethingTheCaseQ() or self.shouldSomethingHappenQ() or
> self.getSomeFlagQ()... I find that's reasonably aesthetically
> pleasing, and is a good hint about what the method is for.

If you want it to be more likely to be recognisable to programmers (and
I would advise you to conform these names to PEP 8), you could name it
as ?foo_p? which is computer-science shortcut for ?foo as a predicate?.

    class Foo(object):

        def should_frobnicate_the_widget_p(self):
            if warble(self):
                return True
            return False

        def content_modified_p(self):
            if wibble(self):
                return False
            return True

-- 
 \          ?When I get real bored, I like to drive downtown and get a |
  `\         great parking spot, then sit in my car and count how many |
_o__)                    people ask me if I'm leaving.? ?Steven Wright |
Ben Finney



From ncoghlan at gmail.com  Thu Aug  6 14:29:31 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 06 Aug 2009 22:29:31 +1000
Subject: [Python-ideas] (try-except) conditional expression similar
 to	(if-else) conditional (PEP 308)
In-Reply-To: <d8cc6f9d0908060503w163fc43dha7c3e7df086aebc2@mail.gmail.com>
References: <d8cc6f9d0908060503w163fc43dha7c3e7df086aebc2@mail.gmail.com>
Message-ID: <4A7ACCAB.3030005@gmail.com>

(moving to python-ideas where this thread belongs)

ilya wrote:
> and I think this can be done just as easily with existing syntax:
> 
>     x = try_1(float, string, except_ = float('nan'), if_ = ValueError)
>     y = try_2(float, string, { ValueError: float('nan') })

As with PEP 308, the thing missing from function based solutions is lazy
evaluation of the operands. They also tend be seriously ugly, as is the
case with the above suggestions.

Given Guido's reluctance in adding the far more common conditional
expressions, I suspect this idea isn't ever going to get anywhere, but
even if that is the case it might be nice to have a rejected PEP that
makes this explicit.

If we were going to do it, I would definitely favour the "EXPR except
ALT_EXPR if EXCEPTION" format, but I doubt it is actually going to get
that far.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ncoghlan at gmail.com  Thu Aug  6 14:32:01 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 06 Aug 2009 22:32:01 +1000
Subject: [Python-ideas] (try-except) conditional expression similar
 to	(if-else) conditional (PEP 308)
In-Reply-To: <e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>
References: <4A7A0626.5050303@lanl.gov>	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>	<20090806012101.B42493A406B@sparrow.telecommunity.com>	<4A7AB4D1.6070501@gmail.com>
	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>
Message-ID: <4A7ACD41.6030605@gmail.com>

(moving to python-ideas)

Dj Gilcrease wrote:
> On Thu, Aug 6, 2009 at 4:47 AM, Nick Coghlan<ncoghlan at gmail.com> wrote:
>> Option 2:
>>  x = float(string) except ValueError: float('nan')
>>  op(float(string) except ValueError: float('nan'))
>>
>> This has the virtue of closely matching the statement syntax, but
>> embedding colons inside expressions is somewhat ugly. Yes, lambda
>> already does it, but lambda can hardly be put forward as a paragon of
>> beauty.
> 
> +1 on this option as it resembles the standard try/except block enough
> it would be a quick edit to convert it to one if later you realize you
> need to catch more exceptions*
> 
> * I recommend NOT allowing multiple exceptions in this form eg
> x = float(string)/var except ValueError, ZeroDivisionError, ...: float('nan')
> 
> as it will start to reduce readability quickly

All 3 components would just be ordinary expressions. The exception
definition would be allowed to resolve to a single exception or a tuple
of exceptions, just as it is in a normal try/except statement.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From digitalxero at gmail.com  Thu Aug  6 15:21:33 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Thu, 6 Aug 2009 07:21:33 -0600
Subject: [Python-ideas] (try-except) conditional expression similar to
	(if-else) conditional (PEP 308)
In-Reply-To: <4A7ACD41.6030605@gmail.com>
References: <4A7A0626.5050303@lanl.gov>
	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>
	<20090806012101.B42493A406B@sparrow.telecommunity.com>
	<4A7AB4D1.6070501@gmail.com>
	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>
	<4A7ACD41.6030605@gmail.com>
Message-ID: <e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>

I figure I would write up the PEP draft, I have never tried writing a
pep before, but i did read PEP 1 and tried to follow it's formating
guides. If there are no additions to the idea, then it seems there
just needs to be a consensus on the syntax before submitting it to the
peps list



  PEP: <pep number>
  Title: try-except conditional expressions
  Version: <svn version string>
  Last-Modified: <svn date string>
  Author: Jeff McAninch <mcaninch at lanl.gov>, Dj Gilcrease
<digitalxero at gmail.com>
  Discussions-To: python-ideas at python.org
  Status: Draft
  Type: Standards Track
  Content-Type: text/plain
  Created: 06-Aug-2009
  Python-Version: 2.7/3.2
  Post-History: <dates of postings to python-list and python-dev>

Abstract:
    I very often want something like a try-except conditional
    expression similar to the if-else conditional instead of resorting
    to a multi-line try-except block.

Design Goals:
    The new syntax should
        * Be simple to read
        * Be intuitive so people who may use it infrequently dont need
            to go lookup the format every time
        * Make it obvious what is happening

Modivation:
    Often when doing calculations or string recasting (to int, float,
    etc) it is required to wrap the section in a simple try-except
    where the exception just assigns a default value. It would be more
    readable and consise if these type of try-excepts could be written
    on a single line.

Issues:
    Unknown

Specification:
    All 3 components would just be ordinary expressions. The exception
    definition would be allowed to resolve to a single exception or a
    tuple of exceptions, just as it is in a normal try/except
    statement.

Syntax Ideas:
    Option 1:
        x = float(string) except float('nan') if ValueError
        op(float(string) except float('nan') if ValueError)

    Option 2:
        x = float(string) except ValueError: float('nan')
        op(float(string) except ValueError: float('nan'))

    Option 3:
        x = float(string) except ValueError else float('nan')
        op(float(string) except ValueError else float('nan'))


From steve at pearwood.info  Thu Aug  6 17:14:15 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 7 Aug 2009 01:14:15 +1000
Subject: [Python-ideas] [Python-Dev] (try-except) conditional expression
	similar =?iso-8859-1?q?to=09?=(if-else) conditional (PEP 308)
In-Reply-To: <loom.20090806T112718-891@post.gmane.org>
References: <4A7A0626.5050303@lanl.gov>
	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>
	<loom.20090806T112718-891@post.gmane.org>
Message-ID: <200908070114.15798.steve@pearwood.info>

Moved to python-ideas.

On Thu, 6 Aug 2009 09:32:16 pm Antoine Pitrou wrote:

> I have to say that the original example:
>     x = float(string) except ValueError else float('nan')
>
> looks artificial. I don't see how it's adequate behaviour to return a
> NaN when presented with a string which doesn't represent a float
> number.

By memory, the IEEE standard allows for 254 different NANs. Back in the 
80s or 90s, SANE (Standard Apple Numerics Environment) supported 
different NANs for different errors, and I'm pretty sure that one of 
them was for failed string-to-float conversions. In any case, the idea 
seems perfectly sensible to me.

As for the suggestion, I'm not convinced by the examples given either. 
I'm not a huge fan of the `x if y else z` construct -- I thought I 
would be, but in practice I nearly never use it -- and I think this is 
just more of the same. Seems good in principle, but in practice, I'm 
not sure it is actually that readable, and there are usually perfectly 
adequate alternatives.

-0 from me.



-- 
Steven D'Aprano


From veloso at verylowsodium.com  Thu Aug  6 17:16:34 2009
From: veloso at verylowsodium.com (Greg Falcon)
Date: Thu, 6 Aug 2009 11:16:34 -0400
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
Message-ID: <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>

On Wed, Aug 5, 2009 at 9:01 PM, Tennessee
Leeuwenburg<tleeuwenburg at gmail.com> wrote:
> def shouldSomethingHappen?(self, context):

> I just think it looks nicer, and is a good hint that a true/false or other
> kind of flag is likely to come back.

Two problems I immediately see with this:

1) '?' is punctuation, and uninitiated readers are likely to guess
that it is some sort of postfix operator.

2) IPython gives a special meaning to ?, and it would be a shame to
collide with that.

Greg F


From dickinsm at gmail.com  Thu Aug  6 17:41:52 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 6 Aug 2009 16:41:52 +0100
Subject: [Python-ideas] [Python-Dev] (try-except) conditional expression
	similar to (if-else) conditional (PEP 308)
In-Reply-To: <200908070114.15798.steve@pearwood.info>
References: <4A7A0626.5050303@lanl.gov>
	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>
	<loom.20090806T112718-891@post.gmane.org>
	<200908070114.15798.steve@pearwood.info>
Message-ID: <5c6f2a5d0908060841p22de821cj98cc6d8c3eee7741@mail.gmail.com>

On Thu, Aug 6, 2009 at 4:14 PM, Steven D'Aprano<steve at pearwood.info> wrote:
> By memory, the IEEE standard allows for 254 different NANs.

Did you mean 2**54 here?  For IEEE 754 binary64 format, I count
2**53-2 bit patterns representing NaNs:  2**52 quiet NaNs and
2**52-2 signaling NaNs. (The 11 exponent bits must all be 1s;
the sign bit and the 52 significand bits can be chosen arbitrarily,
except that two bit patterns are reserved for +/-infinity; the top
significand bit determines whether you've got a signaling or
quiet NaN).

</nitpick>

I agree that that particular example isn't particularly compelling,
but I've definitely encountered other cases where I would have
used this construct if it were available.  Not that that necessarily
makes it a good idea.  :-)

Mark


From lucaspradomelo at gmail.com  Thu Aug  6 20:11:44 2009
From: lucaspradomelo at gmail.com (Lucas Prado Melo)
Date: Thu, 6 Aug 2009 15:11:44 -0300
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
Message-ID: <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>

On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon <veloso at verylowsodium.com>wrote:

> On Wed, Aug 5, 2009 at 9:01 PM, Tennessee
> Leeuwenburg<tleeuwenburg at gmail.com> wrote:
> > def shouldSomethingHappen?(self, context):
>
> > I just think it looks nicer, and is a good hint that a true/false or
> other
> > kind of flag is likely to come back.
>
> Two problems I immediately see with this:
>
> 1) '?' is punctuation, and uninitiated readers are likely to guess
> that it is some sort of postfix operator.

Actually, Ruby allows this notation and, until now, I have not seen many
complains about it.
Despite that, its use would be pretty clear for the beginner, since it would
be associated with meaningful identifiers:

oil_tank.isItFull?()

The worst problem could be the useless parenthesis after the '?' sign.


> 2) IPython gives a special meaning to ?, and it would be a shame to
> collide with that.

 IMHO, IPython might not be mature enough to interfere with Python design
decisions yet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090806/9e182aa6/attachment.html>

From guido at python.org  Thu Aug  6 20:57:25 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 6 Aug 2009 11:57:25 -0700
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> 
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> 
	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
Message-ID: <ca471dc20908061157y22364fc3o52ee0e451eb95d55@mail.gmail.com>

Adding '?' to identifiers in Python ain't gonna happen.

On Thu, Aug 6, 2009 at 11:11 AM, Lucas Prado
Melo<lucaspradomelo at gmail.com> wrote:
> On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon <veloso at verylowsodium.com>
> wrote:
>>
>> On Wed, Aug 5, 2009 at 9:01 PM, Tennessee
>> Leeuwenburg<tleeuwenburg at gmail.com> wrote:
>> > def shouldSomethingHappen?(self, context):
>>
>> > I just think it looks nicer, and is a good hint that a true/false or
>> > other
>> > kind of flag is likely to come back.
>>
>> Two problems I immediately see with this:
>>
>> 1) '?' is punctuation, and uninitiated readers are likely to guess
>> that it is some sort of postfix operator.
>
> Actually, Ruby allows this notation and, until now, I have not seen many
> complains about it.
> Despite that, its use would be pretty clear for the beginner, since it would
> be associated with meaningful identifiers:
>
> oil_tank.isItFull?()
>
> The worst problem could be the useless parenthesis after the '?' sign.
>
>>
>> 2) IPython gives a special meaning to ?, and it would be a shame to
>> collide with that.
>
> ?IMHO, IPython might not be mature enough to interfere with Python design
> decisions yet.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From jnoller at gmail.com  Thu Aug  6 23:07:28 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 6 Aug 2009 17:07:28 -0400
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <ca471dc20908061157y22364fc3o52ee0e451eb95d55@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
	<ca471dc20908061157y22364fc3o52ee0e451eb95d55@mail.gmail.com>
Message-ID: <4222a8490908061407r2975f565tfebfce38acbf7d6f@mail.gmail.com>

+100

On Thu, Aug 6, 2009 at 2:57 PM, Guido van Rossum<guido at python.org> wrote:
> Adding '?' to identifiers in Python ain't gonna happen.
>
> On Thu, Aug 6, 2009 at 11:11 AM, Lucas Prado
> Melo<lucaspradomelo at gmail.com> wrote:
>> On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon <veloso at verylowsodium.com>
>> wrote:
>>>
>>> On Wed, Aug 5, 2009 at 9:01 PM, Tennessee
>>> Leeuwenburg<tleeuwenburg at gmail.com> wrote:
>>> > def shouldSomethingHappen?(self, context):
>>>
>>> > I just think it looks nicer, and is a good hint that a true/false or
>>> > other
>>> > kind of flag is likely to come back.
>>>
>>> Two problems I immediately see with this:
>>>
>>> 1) '?' is punctuation, and uninitiated readers are likely to guess
>>> that it is some sort of postfix operator.
>>
>> Actually, Ruby allows this notation and, until now, I have not seen many
>> complains about it.
>> Despite that, its use would be pretty clear for the beginner, since it would
>> be associated with meaningful identifiers:
>>
>> oil_tank.isItFull?()
>>
>> The worst problem could be the useless parenthesis after the '?' sign.
>>
>>>
>>> 2) IPython gives a special meaning to ?, and it would be a shame to
>>> collide with that.
>>
>> ?IMHO, IPython might not be mature enough to interfere with Python design
>> decisions yet.
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From zuo at chopin.edu.pl  Fri Aug  7 02:15:56 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Fri, 07 Aug 2009 02:15:56 +0200
Subject: [Python-ideas] (try-except) conditional expression similar to
 (if-else) conditional (PEP 308)
In-Reply-To: <e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>
References: <4A7A0626.5050303@lanl.gov>
	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>
	<20090806012101.B42493A406B@sparrow.telecommunity.com>
	<4A7AB4D1.6070501@gmail.com>
	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>
	<4A7ACD41.6030605@gmail.com>
	<e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>
Message-ID: <op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>

> Syntax Ideas:
>     Option 1:
>         x = float(string) except float('nan') if ValueError
>         op(float(string) except float('nan') if ValueError)

-0.5
I wouldn't mix up 'if' word with 'except' -- there are different concepts.

>     Option 2:
>         x = float(string) except ValueError: float('nan')
>         op(float(string) except ValueError: float('nan'))

+0.5
IMHO seems to be the best of the propositions. I'd add to it optional
as (like in present 'try:... except Exc as exc:...')

>     Option 3:
>         x = float(string) except ValueError else float('nan')
>         op(float(string) except ValueError else float('nan'))

-1
IMHO it's illogical. 'else' has completely opposite role in present
'try...except...' clauses. It'd be terribly missleading, not only
for newbies.


Cheers,
*j

-- 
Jan Kaliszewski (zuo) <zuo at chopin.edu.pl>


From jkwon.work at gmail.com  Fri Aug  7 02:28:23 2009
From: jkwon.work at gmail.com (Jae Kwon)
Date: Thu, 6 Aug 2009 17:28:23 -0700
Subject: [Python-ideas] (try-except) conditional expression similar to
	(if-else) conditional (PEP 308)
In-Reply-To: <op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>
References: <4A7A0626.5050303@lanl.gov>
	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>
	<20090806012101.B42493A406B@sparrow.telecommunity.com>
	<4A7AB4D1.6070501@gmail.com>
	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>
	<4A7ACD41.6030605@gmail.com>
	<e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>
	<op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <19461FAF-6637-4769-9415-68CA60BDE059@gmail.com>


On Aug 6, 2009, at 5:15 PM, Jan Kaliszewski wrote:

>> Syntax Ideas:
>>    Option 1:
>>        x = float(string) except float('nan') if ValueError
>>        op(float(string) except float('nan') if ValueError)
>
> -0.5
> I wouldn't mix up 'if' word with 'except' -- there are different  
> concepts.
>
>>    Option 2:
>>        x = float(string) except ValueError: float('nan')
>>        op(float(string) except ValueError: float('nan'))
>
> +0.5
> IMHO seems to be the best of the propositions. I'd add to it optional
> as (like in present 'try:... except Exc as exc:...')
>

+0.5

the colon implies EOL. how about 'then'?

x = float(string) except ValueError [, e] then float('nan')
[, e] optional

  - Jae

>>    Option 3:
>>        x = float(string) except ValueError else float('nan')
>>        op(float(string) except ValueError else float('nan'))
>
> -1
> IMHO it's illogical. 'else' has completely opposite role in present
> 'try...except...' clauses. It'd be terribly missleading, not only
> for newbies.
>
>
> Cheers,
> *j
>
> -- 
> Jan Kaliszewski (zuo) <zuo at chopin.edu.pl>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From python at mrabarnett.plus.com  Fri Aug  7 02:41:08 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 07 Aug 2009 01:41:08 +0100
Subject: [Python-ideas] (try-except) conditional expression similar to
 (if-else) conditional (PEP 308)
In-Reply-To: <op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>
References: <4A7A0626.5050303@lanl.gov>	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>	<20090806012101.B42493A406B@sparrow.telecommunity.com>	<4A7AB4D1.6070501@gmail.com>	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>	<4A7ACD41.6030605@gmail.com>	<e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>
	<op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <4A7B7824.2060504@mrabarnett.plus.com>

Jan Kaliszewski wrote:
>> Syntax Ideas:
>>     Option 1:
>>         x = float(string) except float('nan') if ValueError
>>         op(float(string) except float('nan') if ValueError)
> 
> -0.5
> I wouldn't mix up 'if' word with 'except' -- there are different concepts.
> 
>>     Option 2:
>>         x = float(string) except ValueError: float('nan')
>>         op(float(string) except ValueError: float('nan'))
> 
> +0.5
> IMHO seems to be the best of the propositions. I'd add to it optional
> as (like in present 'try:... except Exc as exc:...')
> 
Can there be multiple 'except's?

     reciprocal = 1.0 / float(string) except ValueError: float('nan') 
except ZeroDivisionError: float('inf')

(We can already chain '==', etc.)

>>     Option 3:
>>         x = float(string) except ValueError else float('nan')
>>         op(float(string) except ValueError else float('nan'))
> 
> -1
> IMHO it's illogical. 'else' has completely opposite role in present
> 'try...except...' clauses. It'd be terribly missleading, not only
> for newbies.
> 


From ben+python at benfinney.id.au  Fri Aug  7 03:39:50 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 07 Aug 2009 11:39:50 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
	<ca471dc20908061157y22364fc3o52ee0e451eb95d55@mail.gmail.com>
Message-ID: <87bpmsmmft.fsf@benfinney.id.au>

Guido van Rossum <guido at python.org> writes:

> Adding '?' to identifiers in Python ain't gonna happen.

Thank you. (Both for making the decision clearly, and the outcome of
that decision.)

-- 
 \      ?Earth gets its price for what Earth gives us.? ?James Russell |
  `\                                                            Lowell |
_o__)                                                                  |
Ben Finney



From stephen at xemacs.org  Fri Aug  7 05:07:09 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Aug 2009 12:07:09 +0900
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
Message-ID: <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp>

Lucas Prado Melo writes:
 > On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon <veloso at verylowsodium.com>wrote:
 > 
 > > On Wed, Aug 5, 2009 at 9:01 PM, Tennessee
 > > Leeuwenburg<tleeuwenburg at gmail.com> wrote:
 > > > def shouldSomethingHappen?(self, context):

 > Actually, Ruby allows this notation and, until now, I have not seen many
 > complains about it.

So does Lisp.  But those languages simply allow ? as part of an identifier.

Python, for one thing, envisions syntax extensions.  Sometimes these
are done with keywords, of course, but sometimes it makes sense to use
an operator (as with decorators' use of "@").  Reserving all
punctuation for this purpose is a reasonable heuristic.

I suspect that the BDFL pronounced simply on the basis that there are
plenty of alternatives (trailing "Q" as Tennessee suggested, trailing
"_p" per Ben) so the benefit is minute, while the costs would be
substantial.

 > > 2) IPython gives a special meaning to ?, and it would be a shame to
 > > collide with that.
 > 
 >  IMHO, IPython might not be mature enough to interfere with Python design
 > decisions yet.

It already has, IIRC.  ISTR IPython was a consideration, though not a
showstopper, in the choice of "@" for decorators.



From lucaspradomelo at gmail.com  Fri Aug  7 05:07:13 2009
From: lucaspradomelo at gmail.com (Lucas Prado Melo)
Date: Fri, 7 Aug 2009 00:07:13 -0300
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
	<871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <6217e0a0908062007x21d0a2afu72fe0c5747778e02@mail.gmail.com>

On Fri, Aug 7, 2009 at 12:07 AM, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> I suspect that the BDFL pronounced simply on the basis that there are
> plenty of alternatives (trailing "Q" as Tennessee suggested, trailing
> "_p" per Ben) so the benefit is minute, while the costs would be
> substantial.

I guess so. I really didn't like the interrogation mark followed by
parenthesis anyway.


>   > > 2) IPython gives a special meaning to ?, and it would be a shame to
>  > > collide with that.
>  >
>  >  IMHO, IPython might not be mature enough to interfere with Python
> design
>  > decisions yet.
>
> It already has, IIRC.  ISTR IPython was a consideration, though not a
> showstopper, in the choice of "@" for decorators.
>
> Oh. Nevermind, then. I thought it wasn't mature enough because the current
stable version has such a low value as 0.10.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090807/d3032b61/attachment.html>

From debatem1 at gmail.com  Fri Aug  7 04:57:53 2009
From: debatem1 at gmail.com (CTO)
Date: Thu, 6 Aug 2009 19:57:53 -0700 (PDT)
Subject: [Python-ideas] Meta idea - monthly digest of python-ideas
In-Reply-To: <6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com>
References: <op.ux5a9nur1q4jlt@whymac>
	<6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com>
Message-ID: <d7abc579-443c-4175-acbe-bf2dee0787de@r27g2000vbn.googlegroups.com>



On Aug 4, 1:58?pm, Michael Foord <fuzzy... at gmail.com> wrote:
> 2009/8/4 Sridhar Ratnakumar <sridh... at activestate.com>
>
> > 1. Create a blog
> > 2. Create a summary of each thread
> > 3. On every month, post the summaries in the blog
>
> > The summary of a thread may adhere to a standard template, for instance,
> > definition of the idea, its merits, and criticisms.
>
> It's a *great* idea, but without someone (or preferably plural) to take it
> on it won't go anywhere.
>
> Michael

I'm happy to contribute. Where do we want the site?

Geremy Condra


From g.brandl at gmx.net  Fri Aug  7 08:26:47 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 07 Aug 2009 08:26:47 +0200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <6217e0a0908062007x21d0a2afu72fe0c5747778e02@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>	<871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<6217e0a0908062007x21d0a2afu72fe0c5747778e02@mail.gmail.com>
Message-ID: <h5ghhl$e4$1@ger.gmane.org>

Lucas Prado Melo schrieb:

>      > > 2) IPython gives a special meaning to ?, and it would be a shame to
>      > > collide with that.
>      >
>      >  IMHO, IPython might not be mature enough to interfere with
>     Python design
>      > decisions yet.
> 
>     It already has, IIRC.  ISTR IPython was a consideration, though not a
>     showstopper, in the choice of "@" for decorators.
> 
> Oh. Nevermind, then. I thought it wasn't mature enough because the
> current stable version has such a low value as 0.10.

Don't judge open source projects' maturity by their version number.  For some
reason, authors have a tendency to be over-modest in switching to something
post-0.x, even if the project is stable and well-used for years.

It's probably just a reaction to the orthogonal practice of some corporate
software vendors :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From g.brandl at gmx.net  Fri Aug  7 08:30:14 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 07 Aug 2009 08:30:14 +0200
Subject: [Python-ideas] (try-except) conditional expression similar to
 (if-else) conditional (PEP 308)
In-Reply-To: <4A7B7824.2060504@mrabarnett.plus.com>
References: <4A7A0626.5050303@lanl.gov>	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>	<20090806012101.B42493A406B@sparrow.telecommunity.com>	<4A7AB4D1.6070501@gmail.com>	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>	<4A7ACD41.6030605@gmail.com>	<e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>	<op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>
	<4A7B7824.2060504@mrabarnett.plus.com>
Message-ID: <h5gho4$e4$2@ger.gmane.org>

MRAB schrieb:
> Jan Kaliszewski wrote:
>>> Syntax Ideas:
>>>     Option 1:
>>>         x = float(string) except float('nan') if ValueError
>>>         op(float(string) except float('nan') if ValueError)
>> 
>> -0.5
>> I wouldn't mix up 'if' word with 'except' -- there are different concepts.
>> 
>>>     Option 2:
>>>         x = float(string) except ValueError: float('nan')
>>>         op(float(string) except ValueError: float('nan'))
>> 
>> +0.5
>> IMHO seems to be the best of the propositions. I'd add to it optional
>> as (like in present 'try:... except Exc as exc:...')
>> 
> Can there be multiple 'except's?
> 
>      reciprocal = 1.0 / float(string) except ValueError: float('nan') 
> except ZeroDivisionError: float('inf')

Don't over-complicate.  This proposal has a slim chance of success as it
is; piling on top isn't going to help.

IMO, any syntax that puts something other than the exception class after
"except" has zero chance.  The "except-then" is the most attractive to
me, but still doesn't read as naturally as "if-else".

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From mwm-keyword-python.b4bdba at mired.org  Fri Aug  7 09:21:20 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Fri, 7 Aug 2009 03:21:20 -0400
Subject: [Python-ideas] (try-except) conditional expression similar
	to	(if-else) conditional (PEP 308)
In-Reply-To: <h5gho4$e4$2@ger.gmane.org>
References: <4A7A0626.5050303@lanl.gov>
	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>
	<20090806012101.B42493A406B@sparrow.telecommunity.com>
	<4A7AB4D1.6070501@gmail.com>
	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>
	<4A7ACD41.6030605@gmail.com>
	<e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>
	<op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>
	<4A7B7824.2060504@mrabarnett.plus.com> <h5gho4$e4$2@ger.gmane.org>
Message-ID: <20090807032120.37476799@bhuda.mired.org>

On Fri, 07 Aug 2009 08:30:14 +0200
Georg Brandl <g.brandl at gmx.net> wrote:

> MRAB schrieb:
> > Jan Kaliszewski wrote:
> >>> Syntax Ideas:
> >>>     Option 1:
> >>>         x = float(string) except float('nan') if ValueError
> >>>         op(float(string) except float('nan') if ValueError)
> >> 
> >> -0.5
> >> I wouldn't mix up 'if' word with 'except' -- there are different concepts.
> >> 
> >>>     Option 2:
> >>>         x = float(string) except ValueError: float('nan')
> >>>         op(float(string) except ValueError: float('nan'))
> >> 
> >> +0.5
> >> IMHO seems to be the best of the propositions. I'd add to it optional
> >> as (like in present 'try:... except Exc as exc:...')
> >> 
> > Can there be multiple 'except's?
> > 
> >      reciprocal = 1.0 / float(string) except ValueError: float('nan') 
> > except ZeroDivisionError: float('inf')
> 
> Don't over-complicate.  This proposal has a slim chance of success as it
> is; piling on top isn't going to help.

Assuming you avoid the ':' version, shouldn't multiple excepts fall
out of the grammar naturally?

x = ((1.0 / float(string)
      except float('nan') if ValueError)
     except float('inf') if ZeroDivisionError)

parses as "expr1 except float('inf') if ValueError", where expr1 is 
"(1.0 / float(string) except float('nan') if ValueError)".

Yes, it's as ugly as nesting if/else expressions, but those work, so
these should.

FWIW, I'm +.5 on this. I don't see a lot of use for it, but can see
that where it is useful, it's easier to read than a try:/except:.

     <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From jkwon.work at gmail.com  Fri Aug  7 10:03:02 2009
From: jkwon.work at gmail.com (Jae Kwon)
Date: Fri, 7 Aug 2009 01:03:02 -0700
Subject: [Python-ideas] (try-except) conditional expression similar
	to	(if-else) conditional (PEP 308)
In-Reply-To: <20090807032120.37476799@bhuda.mired.org>
References: <4A7A0626.5050303@lanl.gov>
	<592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1>
	<20090806012101.B42493A406B@sparrow.telecommunity.com>
	<4A7AB4D1.6070501@gmail.com>
	<e9764b730908060418p1e33e4b1k6a3e3c31369e7d8a@mail.gmail.com>
	<4A7ACD41.6030605@gmail.com>
	<e9764b730908060621v7b5632fye07e45ea1dd05e82@mail.gmail.com>
	<op.ux9m0ur2fvx12t@jerzozwiesz.home.aster.pl>
	<4A7B7824.2060504@mrabarnett.plus.com> <h5gho4$e4$2@ger.gmane.org>
	<20090807032120.37476799@bhuda.mired.org>
Message-ID: <1BF02135-CF7D-4541-83F4-7093131704B2@gmail.com>

How about making this work?

foo = try: float(string) except SomethingError, e: float(NAN)

This would imply that try blocks have a value. Maybe even allow this:

foo = try:
     float(string)
except Something Error, e:
     float(NAN)

Sure, it introduces a couple new grammar tweaks, but it's easy to  
understand and is backwards compatible.

  - Jae


On Aug 7, 2009, at 12:21 AM, Mike Meyer wrote:

> On Fri, 07 Aug 2009 08:30:14 +0200
> Georg Brandl <g.brandl at gmx.net> wrote:
>
>> MRAB schrieb:
>>> Jan Kaliszewski wrote:
>>>>> Syntax Ideas:
>>>>>    Option 1:
>>>>>        x = float(string) except float('nan') if ValueError
>>>>>        op(float(string) except float('nan') if ValueError)
>>>>
>>>> -0.5
>>>> I wouldn't mix up 'if' word with 'except' -- there are different  
>>>> concepts.
>>>>
>>>>>    Option 2:
>>>>>        x = float(string) except ValueError: float('nan')
>>>>>        op(float(string) except ValueError: float('nan'))
>>>>
>>>> +0.5
>>>> IMHO seems to be the best of the propositions. I'd add to it  
>>>> optional
>>>> as (like in present 'try:... except Exc as exc:...')
>>>>
>>> Can there be multiple 'except's?
>>>
>>>     reciprocal = 1.0 / float(string) except ValueError: float('nan')
>>> except ZeroDivisionError: float('inf')
>>
>> Don't over-complicate.  This proposal has a slim chance of success  
>> as it
>> is; piling on top isn't going to help.
>
> Assuming you avoid the ':' version, shouldn't multiple excepts fall
> out of the grammar naturally?
>
> x = ((1.0 / float(string)
>      except float('nan') if ValueError)
>     except float('inf') if ZeroDivisionError)
>
> parses as "expr1 except float('inf') if ValueError", where expr1 is
> "(1.0 / float(string) except float('nan') if ValueError)".
>
> Yes, it's as ugly as nesting if/else expressions, but those work, so
> these should.
>
> FWIW, I'm +.5 on this. I don't see a lot of use for it, but can see
> that where it is useful, it's easier to read than a try:/except:.
>
>     <mike
> -- 
> Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
> Independent Network/Unix/Perforce consultant, email for more  
> information.
>
> O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From greg.ewing at canterbury.ac.nz  Fri Aug  7 11:10:24 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 07 Aug 2009 21:10:24 +1200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
	<871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4A7BEF80.1090505@canterbury.ac.nz>

Stephen J. Turnbull wrote:

> So does Lisp.  But those languages simply allow ? as part of an identifier.

Lisp allows almost *anything* as part of an identifier.

Python doesn't, and I like it that way. Making exceptions
for just a few characters would make the rules for what
is allowed in an identifier more complicated and harder
to remember, for little benefit that I can see.

-- 
Greg


From ede at mit.edu  Fri Aug  7 12:04:47 2009
From: ede at mit.edu (Eric Eisner)
Date: Fri, 7 Aug 2009 19:04:47 +0900
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
Message-ID: <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>

Hello,

As previously mentioned on python-ideas [1] (circa 2006), it would
make sense to be able to perform bitwise operations on
bytes/bytearray.

Stealing the example from the original suggestion:

Suppose I have a string (say, read in from a binary file) that has a
4-byte field in it. Bit 11 indicates whether the accompanying data is
in glorped form (something I'm just making up for this example).

For example, if the field has 0x000480c4, the data is not glorped. If
the data is glorped, bit 11 would be on, i.e., 0x001480c4.

Let's say I want to turn on the glorp bit; what I have to do now:

GLORPED = 0x10
newchar = flags[1] | GLORPED
flags = flags[0] + newchar + flags[2:]

What I'd like to be able to do is something like:

GLORPED = b"\x00\x10\x00\x00"
flags |= GLORPED

# test if the glorped bit is on
any(flags & GLORPED)


I have run into this a few times, at least when reading/writing binary
formats etc. This approach is more intuitive than the typical/archaic
way of converting to an integer, performing a bitwise operation on the
integer, converting back to bytes. Arguably, bitwise operations on a
high-level integer type don't make sense, as base 2 is an
implementation detail.

At the very least, bytes and bytearray should be usable with the ~ ^ |
& operators

Example behavior:

>>> ~b'\x55\xff'
b'\xaa\x00'
>>> b'\xf5\x60' & b'\xa9\x3c'
b'\xa1\x20'

Unresolved problems:
If the two arguments are of different length, either it could either
raise a ValueError or mimic the behavior of ints.
Xoring an int to a byte seems less than well defined in general, due
to endianness ambiguity of the int and size ambiguity. I would think
this should not be allowed.
Also conceivable is using the shift operators >> and << on bytes, but
I personally would use that less often, and the result of such an
operation is ambiguous due to endianness.


-Eric Eisner

[1] http://mail.python.org/pipermail/python-ideas/2006-December/000001.html


From taleinat at gmail.com  Fri Aug  7 13:43:54 2009
From: taleinat at gmail.com (Tal Einat)
Date: Fri, 7 Aug 2009 14:43:54 +0300
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
Message-ID: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>

Eric Eisner wrote:
> Hello,
>
> As previously mentioned on python-ideas [1] (circa 2006), it would
> make sense to be able to perform bitwise operations on
> bytes/bytearray.
>
> Stealing the example from the original suggestion:
>
> Suppose I have a string (say, read in from a binary file) that has a
> 4-byte field in it. Bit 11 indicates whether the accompanying data is
> in glorped form (something I'm just making up for this example).
>
> For example, if the field has 0x000480c4, the data is not glorped. If
> the data is glorped, bit 11 would be on, i.e., 0x001480c4.
>
> Let's say I want to turn on the glorp bit; what I have to do now:
>
> GLORPED = 0x10
> newchar = flags[1] | GLORPED
> flags = flags[0] + newchar + flags[2:]
>
> What I'd like to be able to do is something like:
>
> GLORPED = b"\x00\x10\x00\x00"
> flags |= GLORPED

You can already do bitwise operations on bytes, and retrieving the
relevant byte and using single-byte operations on it is simple enough.
As I see it, the major underlying problem here is that byte-arrays are
immutable, since what you really want is to be able to change a single
byte of the array in-place. In general, Python's byte arrays and
strings are very ill suited for dealing with data which isn't
byte-based, and definitely horrible at building or modifying data
streams.

I recommend using an external library for working with data structures
and/or data streams. Construct[1] is my personal favorite, since it's
especially Pythonic and easy to use.

In any case, if you wish to propose a mutable byte/bit array which
supports array-wise binary operators, I can say I would certainly be
glad to have such a class at my disposal.

- Tal

[1] http://construct.wikispaces.com/


From phd at phd.pp.ru  Fri Aug  7 13:51:08 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Fri, 7 Aug 2009 15:51:08 +0400
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>
Message-ID: <20090807115108.GA32694@phd.pp.ru>

On Fri, Aug 07, 2009 at 02:43:54PM +0300, Tal Einat wrote:
> In any case, if you wish to propose a mutable byte/bit array which
> supports array-wise binary operators, I can say I would certainly be
> glad to have such a class at my disposal.

   Like this http://pypi.python.org/pypi/BitVector/ ?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.


From ede at mit.edu  Fri Aug  7 13:57:54 2009
From: ede at mit.edu (Eric Eisner)
Date: Fri, 7 Aug 2009 20:57:54 +0900
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <20090807115108.GA32694@phd.pp.ru>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>
	<20090807115108.GA32694@phd.pp.ru>
Message-ID: <42ef4ee0908070457j32ff805fy3a4ef6dd350e9c39@mail.gmail.com>

On Fri, Aug 7, 2009 at 20:51, Oleg Broytmann<phd at phd.pp.ru> wrote:
> On Fri, Aug 07, 2009 at 02:43:54PM +0300, Tal Einat wrote:
>> In any case, if you wish to propose a mutable byte/bit array which
>> supports array-wise binary operators, I can say I would certainly be
>> glad to have such a class at my disposal.
>
> ? Like this http://pypi.python.org/pypi/BitVector/ ?

BitVector does seem to nicely handle every conceivable usecase I had
in mind, but I think some of that functionality would be useful to
have available in the standard implementation of bytes.

-Eric


From ncoghlan at gmail.com  Fri Aug  7 14:24:01 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 07 Aug 2009 22:24:01 +1000
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>
Message-ID: <4A7C1CE1.4080307@gmail.com>

Tal Einat wrote:
> You can already do bitwise operations on bytes, and retrieving the
> relevant byte and using single-byte operations on it is simple enough.
> As I see it, the major underlying problem here is that byte-arrays are
> immutable, since what you really want is to be able to change a single
> byte of the array in-place. In general, Python's byte arrays and
> strings are very ill suited for dealing with data which isn't
> byte-based, and definitely horrible at building or modifying data
> streams.

Careful with your terminology here. In Python 3.x, the following two
types both exist:

bytes: immutable sequence of integers between 0 and 255 inclusive
bytearray: similar to bytes, but mutable

The operations ~x, x&y, x|y and x^y make sense for both bytes and bytearray.

The augmented assignment operators only make sense for bytearray.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ilya.nikokoshev at gmail.com  Fri Aug  7 14:46:40 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Fri, 7 Aug 2009 16:46:40 +0400
Subject: [Python-ideas] pep 312 - implicit lambdas idea
Message-ID: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>

I was thinking about a good syntax for implicit lambdas for a while
and today I had this idea: make ``_:`` a shortcut for ``lambda
_=None:``

For example:

    map( _: _ + 5, some_list)
    register_callback( _: True)
    def apply_transform(..., transform = _:_, ... ):

but still

    addition = lamba x, y: x + y

The rationale is that you only want to get rid of lambda keyword to
create a *very* simple function, the one that will be called either
without parameters or with only one parameter. For everything more
complicated, you really should go and write the explicit function
signature using lambda.

Even though ``:`` could theoretically denote implicit lambda, it's too
easy to miss it. The combination ``_:`` is much easier to notice. It
also makes explicit that there is at most one parameter and it's name
is ``_``. Since it's very short, it can easily be used in a long
function call or as a default parameter, as above

Your thoughts?


From gerald.britton at gmail.com  Fri Aug  7 14:58:28 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Fri, 7 Aug 2009 08:58:28 -0400
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
Message-ID: <5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com>

The underscore is already a viable identifier:
>>> _ = lambda x:x*2
>>> _(3)
6

and

>>> x = lambda _:_*2
>>> x(3)
6
>>>


So I don't think you can use it the way you're proposing without breaking
existing programs


On Fri, Aug 7, 2009 at 8:46 AM, ilya <ilya.nikokoshev at gmail.com> wrote:

> I was thinking about a good syntax for implicit lambdas for a while
> and today I had this idea: make ``_:`` a shortcut for ``lambda
> _=None:``
>
> For example:
>
>    map( _: _ + 5, some_list)
>    register_callback( _: True)
>    def apply_transform(..., transform = _:_, ... ):
>
> but still
>
>    addition = lamba x, y: x + y
>
> The rationale is that you only want to get rid of lambda keyword to
> create a *very* simple function, the one that will be called either
> without parameters or with only one parameter. For everything more
> complicated, you really should go and write the explicit function
> signature using lambda.
>
> Even though ``:`` could theoretically denote implicit lambda, it's too
> easy to miss it. The combination ``_:`` is much easier to notice. It
> also makes explicit that there is at most one parameter and it's name
> is ``_``. Since it's very short, it can easily be used in a long
> function call or as a default parameter, as above
>
> Your thoughts?
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090807/20f36c16/attachment.html>

From ilya.nikokoshev at gmail.com  Fri Aug  7 15:08:38 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Fri, 7 Aug 2009 17:08:38 +0400
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com>
Message-ID: <d8cc6f9d0908070608u74a83ac2j2fb45083316e459a@mail.gmail.com>

I'd like to use this syntax only in situations of PEP 312, that is,
where a colon is prohibited by current grammar. In those situations,
writing _: is also prohibited by current grammar so it will be parsed
unambiguously.

The examples you quoted will not change under my idea, but ``x =
(_:_*2)`` would be possible as an alternative. Note that I would be
against being able to use ``x = _:_*2``. I think it's better to
require either lambda keyword or parentheses for the RHS of the
assignment as colon has a well-defined meaning as starting an indented
block unless it's inside some brackets.

On Fri, Aug 7, 2009 at 4:58 PM, Gerald Britton<gerald.britton at gmail.com> wrote:
> The underscore is already a viable identifier:
>>>> _ = lambda x:x*2
>>>> _(3)
> 6
> and
>>>> x = lambda _:_*2
>>>> x(3)
> 6
>>>>
>
> So I don't think you can use it the way you're proposing without breaking
> existing programs
>
> On Fri, Aug 7, 2009 at 8:46 AM, ilya <ilya.nikokoshev at gmail.com> wrote:
>>
>> I was thinking about a good syntax for implicit lambdas for a while
>> and today I had this idea: make ``_:`` a shortcut for ``lambda
>> _=None:``
>>
>> For example:
>>
>> ? ?map( _: _ + 5, some_list)
>> ? ?register_callback( _: True)
>> ? ?def apply_transform(..., transform = _:_, ... ):
>>
>> but still
>>
>> ? ?addition = lamba x, y: x + y
>>
>> The rationale is that you only want to get rid of lambda keyword to
>> create a *very* simple function, the one that will be called either
>> without parameters or with only one parameter. For everything more
>> complicated, you really should go and write the explicit function
>> signature using lambda.
>>
>> Even though ``:`` could theoretically denote implicit lambda, it's too
>> easy to miss it. The combination ``_:`` is much easier to notice. It
>> also makes explicit that there is at most one parameter and it's name
>> is ``_``. Since it's very short, it can easily be used in a long
>> function call or as a default parameter, as above
>>
>> Your thoughts?
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
>
>
> --
> Gerald Britton
>


From dickinsm at gmail.com  Fri Aug  7 15:19:20 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 7 Aug 2009 14:19:20 +0100
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
Message-ID: <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>

On Fri, Aug 7, 2009 at 11:04 AM, Eric Eisner<ede at mit.edu> wrote:
> Hello,
>
> As previously mentioned on python-ideas [1] (circa 2006), it would
> make sense to be able to perform bitwise operations on
> bytes/bytearray.

+1 from me.  I'd say that it makes just as much sense to do bit
operations on bytes or bytearrays as it does on integers.
I've always felt a little bit dirty when I've abused ints as
bitstrings in the past;  byte strings seem better suited to this
task, especially where input and output are also involved.

> Unresolved problems:
> If the two arguments are of different length, either it could either
> raise a ValueError or mimic the behavior of ints.

I'd say ValueError, since it's not really clear what 'mimic the
behaviour of ints' means here.  E.g., should

bytes([1,2,3,4]) | bytes([16, 32, 64])

be equal to bytes([17, 34, 67, 64]) or bytes([1, 18, 35, 68]).
It seems better to restrict the use to the cases where
there's a single obvious interpretation.

> Xoring an int to a byte seems less than well defined in general, due
> to endianness ambiguity of the int and size ambiguity. I would think
> this should not be allowed.

Agreed.

> Also conceivable is using the shift operators >> and << on bytes, but
> I personally would use that less often, and the result of such an
> operation is ambiguous due to endianness.

Agreed.  To make sense of the shift operators you effectively have
to give 'position' interpretations for the individual bits, and there's no
single obvious way of doing this;  for the plain bitwise operations
this isn't necessary.

One other question:  if a has type bytes and b has type bytearray,
what types would you propose that a & b and b & a should have?

Mark


From masklinn at masklinn.net  Fri Aug  7 15:20:19 2009
From: masklinn at masklinn.net (Masklinn)
Date: Fri, 7 Aug 2009 15:20:19 +0200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <d8cc6f9d0908070608u74a83ac2j2fb45083316e459a@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com>
	<d8cc6f9d0908070608u74a83ac2j2fb45083316e459a@mail.gmail.com>
Message-ID: <2E804908-D98C-4935-8EC2-B07CF10EF1EB@masklinn.net>

On 7 Aug 2009, at 15:08 , ilya wrote:
> I'd like to use this syntax only in situations of PEP 312, that is,
> where a colon is prohibited by current grammar. In those situations,
> writing _: is also prohibited by current grammar so it will be parsed
> unambiguously.
>
> The examples you quoted will not change under my idea, but ``x =
> (_:_*2)`` would be possible as an alternative. Note that I would be
> against being able to use ``x = _:_*2``. I think it's better to
> require either lambda keyword or parentheses for the RHS of the
> assignment as colon has a well-defined meaning as starting an indented
> block unless it's inside some brackets.

It's not much of a gain though, some kind of currying for operators  
might be a more general idea for those cases. Or simply a better  
lambda syntax (Haskell has a pretty cool one for what it's worth, but  
it might conflict with the EOL escape thing), but as far as I know  
previous discussions on the subject have either failed to come up with  
a viable syntax or failed to get the approval of the community or BDFL.


From ede at mit.edu  Fri Aug  7 15:31:27 2009
From: ede at mit.edu (Eric Eisner)
Date: Fri, 7 Aug 2009 22:31:27 +0900
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
Message-ID: <42ef4ee0908070631o3f751ff0t59e31ea503dfbbff@mail.gmail.com>

> One other question: ?if a has type bytes and b has type bytearray,
> what types would you propose that a & b and b & a should have?

For consistency they should probably do what they currently do with
concatenation: the type of the first expression is the type of the
result:

>>> b'a' + bytearray(b'b')
b'ab'
>>> bytearray(b'b') + b'a'
bytearray(b'ba')


-Eric


From ben+python at benfinney.id.au  Fri Aug  7 16:06:59 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Sat, 08 Aug 2009 00:06:59 +1000
Subject: [Python-ideas] pep 312 - implicit lambdas idea
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
Message-ID: <87d477lnuk.fsf@benfinney.id.au>

ilya <ilya.nikokoshev at gmail.com> writes:

> I was thinking about a good syntax for implicit lambdas for a while

Why were you thinking of this, and why is it important enough to
override ?explicit is better than implicit??

> and today I had this idea: make ``_:`` a shortcut for ``lambda
> _=None:``

-1.

The name ?_? already sees a lot of use as an identifier. It is in common
use as a conventional shortcut to the ?gettext.gettext? function. It is
also commonly used as a name to bind to a value that is returned from
some process but not actually needed.

Overloading existing conventions of that valid identifier with this
special construct is too much, in my view.

> Even though ``:`` could theoretically denote implicit lambda, it's too
> easy to miss it. The combination ``_:`` is much easier to notice.

I disagree; it is too easy to miss.

-- 
 \        ?Last year I went fishing with Salvador Dali. He was using a |
  `\          dotted line. He caught every other fish.? ?Steven Wright |
_o__)                                                                  |
Ben Finney



From debatem1 at gmail.com  Fri Aug  7 16:46:13 2009
From: debatem1 at gmail.com (CTO)
Date: Fri, 7 Aug 2009 07:46:13 -0700 (PDT)
Subject: [Python-ideas] Meta idea - monthly digest of python-ideas
In-Reply-To: <d7abc579-443c-4175-acbe-bf2dee0787de@r27g2000vbn.googlegroups.com>
References: <op.ux5a9nur1q4jlt@whymac>
	<6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com> 
	<d7abc579-443c-4175-acbe-bf2dee0787de@r27g2000vbn.googlegroups.com>
Message-ID: <74efd982-7ec4-48c2-a02b-b72faf214915@r24g2000vbn.googlegroups.com>



On Aug 6, 10:57?pm, CTO <debat... at gmail.com> wrote:
> On Aug 4, 1:58?pm, Michael Foord <fuzzy... at gmail.com> wrote:
>
> > 2009/8/4 Sridhar Ratnakumar <sridh... at activestate.com>
>
> > > 1. Create a blog
> > > 2. Create a summary of each thread
> > > 3. On every month, post the summaries in the blog
>
> > > The summary of a thread may adhere to a standard template, for instance,
> > > definition of the idea, its merits, and criticisms.
>
> > It's a *great* idea, but without someone (or preferably plural) to take it
> > on it won't go anywhere.
>
> > Michael

Went ahead and created a temporary site for us over at squarespace. If
we
need to move it it shouldn't be hard. Will need at least one other
contributor
to make this work, though.

Geremy Condra


From steve at pearwood.info  Sat Aug  8 02:21:15 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Aug 2009 10:21:15 +1000
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
Message-ID: <200908081021.15580.steve@pearwood.info>

On Fri, 7 Aug 2009 10:46:40 pm ilya wrote:
> I was thinking about a good syntax for implicit lambdas for a while
> and today I had this idea: make ``_:`` a shortcut for ``lambda
> _=None:``

[...]

> The rationale is that you only want to get rid of lambda keyword to
> create a *very* simple function, the one that will be called either
> without parameters or with only one parameter. For everything more
> complicated, you really should go and write the explicit function
> signature using lambda.

Why would you want to get rid of the lambda keyword? What's the benefit?

Is this about saving twelve keystrokes?

lambda _=None:
  versus
_:

Just how often do you want, or need, to write such a lambda? It seems to 
me that not only is it a special case you want to break the rules for, 
which goes against the Zen, but it's an incredibly rare special case.

_ as an identifier already has three conventional meanings:

(1) In the interactive interpreter, _ is the value of the last 
expression typed.

(2) It is commonly used to mean "I don't care about this value", e.g. 

t = ("Fred Smith", "123 Fourth Street", "New York", "dog")
name, _, _, pet = t

(3) It is also often used in internationalization.

You want to give it the extra meaning "a default parameter name for 
lambda when I can't be bothered typing even a single letter name".

Because _ is already a valid identifier, this will break code that does 
this:

while _:
    process()
    _ = function()

if _:
    print "something"


Not the best choice of names, but somebody, somewhere, is doing that, 
and your suggestion will break their code.


Looking at the three examples you gave:

map( _: _ + 5, some_list)
register_callback( _: True)
def apply_transform(..., transform = _:_, ... ):

In the first case, I wouldn't use the short-cut form even if it were 
available. I'd write a lambda that used a more meaningful name. In this 
case, I'm expecting an int, so I would use n, or a float, so I'd use x. 
I'd also avoid setting the pointless default:

map(lambda x: x+5, some_list)
  vs
map(_: _+5, some_list)

Since your suggestion doesn't do precisely what I want, the only reason 
I would have for using your construct is to save seven keystrokes. 
Encouraging laziness on the behalf of the programmer is not a good 
reason for special-casing rare cases.

Second case: register_callback( _: True)

I assume you're implying that the callback function must take a single 
argument. In this example, using _ as the parameter name to the lambda 
makes sense, because it is a "don't care" argument. But if the callback 
function is documented as always being given a single argument, I would 
want to know if it was being called without any arguments, so the 
default value of None is inappropriate and I would avoid using it.

Third case: def apply_transform(..., transform = _:_, ... ):

I don't think I'd write a function called apply_transform() which made 
the transformation function optional, let alone buried deep in the 
middle of a whole lot of extra parameters. (I presume that's what 
the "..."s are meant to imply.) But putting that aside, I see your 
intention: a default do-nothing function which appears in a very long 
parameter list. The problem is that instead of trying to shrink the 
default value so you can fit all the parameters on a single line, you 
should make such a complicated function signature more readable by 
spreading it out:

def apply_transform(
    obj,
    start, end,      # start (inc) and end (exc) positions to apply
    another_arg,     # does something very important I'm sure
    x=0, y=1, z=2,   # more very important arguments  
    transform=(      # default null transformation
              lambda obj=None: obj),  
    frotz=False,     # if true, frotz the hymangirator with spangule
    hymangirator=None,
    spangule=None,
    magic=12345,     # this needs no documentation
    namespace={},
    flibbertigibbet=None,
    _private_magic=[]  # the caller shouldn't supply this
    ):

(Even better is to avoid such complicated function signatures, but 
sometimes that's not an option.)

So again I'd be very unlikely to use your suggested construct except out 
of simple can't-be-bothered-to-type-a-dozen-letters laziness. Pandering 
to that sort of laziness is probably not a good thing.

Fundamentally, this suggestion doesn't add expressability to the 
language, or power. Laziness on it's own is not a good reason for 
special casing rare cases. If it was a very common case, then *perhaps* 
you would have an argument for special casing needless verbiage: 
conciseness (up to a point) is a virtue in a language. That's partly 
why we have lambdas in the first place, so we can write this:

reduce(lambda a,b: (a+b)/2.0, somelist)

instead of this:

def average(a, b):
    return (a+b)/2.0
reduce(average, somelist)

But this isn't a common case, it's a rare case, and the case you're 
hoping to replace is pretty concise already.



-- 
Steven D'Aprano


From greg.ewing at canterbury.ac.nz  Sat Aug  8 02:49:24 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 08 Aug 2009 12:49:24 +1200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <87d477lnuk.fsf@benfinney.id.au>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<87d477lnuk.fsf@benfinney.id.au>
Message-ID: <4A7CCB94.8020408@canterbury.ac.nz>

My suggestion for a lightweight lambda syntax is

   args -> expr

Examples:

   map(x -> x + 5, something)

   d = defauldict(->[])

-- 
Greg



From tleeuwenburg at gmail.com  Sat Aug  8 03:07:30 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Sat, 8 Aug 2009 11:07:30 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <4A7BEF80.1090505@canterbury.ac.nz>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com>
	<6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com>
	<871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4A7BEF80.1090505@canterbury.ac.nz>
Message-ID: <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>

Hi all,
Well, the thread's basically concluded, but I'm not sure if it's appropriate
to just 'leave it hanging' seeing as I instigated the thread. So thanks all
for the discussion points and I think we can wrap things up now. Please feel
free to comment on the summary below if you think it doesn't have all the
relevant details.

To summarise the discussion for posterity:

On the Plus Side:
   ? is part of English grammar, and is an intuitive clue-in that a yes/no,
true/false response is expected as a result. Some people (myself included)
would find this an aesthetically pleasing way to label methods which will
come back with a flag.

On the Negative Side:
  ? looks like an operator, not part of the method name, and might confuse
people
  It wasn't mentioned much, but of course there is all the work involved to
change the Python grammar!
  It could collide with external dependencies such as IPython

Outcome:
  It was decided this would not be adopted.

Secondary outcomes:
  A couple of idioms were suggested, such as a trailing capital Q instead
(in my mind, Q for question) of the ?, or _p for predicate. It's possible
people may find it useful to adopt or further discuss such idioms.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090808/a3c95ae2/attachment.html>

From greg.ewing at canterbury.ac.nz  Sat Aug  8 03:17:39 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 08 Aug 2009 13:17:39 +1200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <200908081021.15580.steve@pearwood.info>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<200908081021.15580.steve@pearwood.info>
Message-ID: <4A7CD233.7060905@canterbury.ac.nz>

Steven D'Aprano wrote:

> Why would you want to get rid of the lambda keyword? What's the benefit?
> 
> Is this about saving twelve keystrokes?

It's about conciseness. The only time it makes sense to
write a function in-line is when the body is extremely
short -- but then it gets swamped by the lambda keyword
itself.

But I agree that the OP's particular solution is restricted
to a case that's too special. My version is completely
general -- it can express anything that the existing lambda
can express.

-- 
Greg


From steve at pearwood.info  Sat Aug  8 03:53:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Aug 2009 11:53:38 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<4A7BEF80.1090505@canterbury.ac.nz>
	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>
Message-ID: <200908081153.39137.steve@pearwood.info>

On Sat, 8 Aug 2009 11:07:30 am Tennessee Leeuwenburg wrote:
> Hi all,
> Well, the thread's basically concluded, but I'm not sure if it's
> appropriate to just 'leave it hanging' seeing as I instigated the
> thread. So thanks all for the discussion points and I think we can
> wrap things up now. Please feel free to comment on the summary below
> if you think it doesn't have all the relevant details.
>
> To summarise the discussion for posterity:
>
> On the Plus Side:
>    ? is part of English grammar, and is an intuitive clue-in that a
> yes/no, true/false response is expected as a result.

What makes you think that the question mark is a clue-in that a yes/no 
answer is expected?

When did you come up with the idea?

Who would agree with you?

Where can I see some evidence for that suggestion?

Excepting rhetorical questions, ? is an intuitive clue-in that a 
response is required, with no limitation on the nature of the response. 
Who/what/why/how/when questions expect open-ended answers. It seems to 
me that the clue-in that a yes/no answer is expected comes from 
existence words and intention words, those related to be and will:

Is there a doctor in the house?
Would you pass me the spanner?
Can I do this?

In any case, I don't think that "returns a boolean flag" versus "returns 
a non-flag" is an important distinction. Flags can be used as arbitrary 
objects, and non-flags can be used as flags.



-- 
Steven D'Aprano


From gerald.britton at gmail.com  Sat Aug  8 07:10:15 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Sat, 8 Aug 2009 01:10:15 -0400
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <4A7CD233.7060905@canterbury.ac.nz>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com> 
	<200908081021.15580.steve@pearwood.info>
	<4A7CD233.7060905@canterbury.ac.nz>
Message-ID: <5d1a32000908072210m493dffd5k1a6ba73fd03999bc@mail.gmail.com>

+1

On Fri, Aug 7, 2009 at 9:17 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Steven D'Aprano wrote:
>
>  Why would you want to get rid of the lambda keyword? What's the benefit?
>>
>> Is this about saving twelve keystrokes?
>>
>
> It's about conciseness. The only time it makes sense to
> write a function in-line is when the body is extremely
> short -- but then it gets swamped by the lambda keyword
> itself.
>
> But I agree that the OP's particular solution is restricted
> to a case that's too special. My version is completely
> general -- it can express anything that the existing lambda
> can express.
>
> --
> Greg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090808/b7ed3b9b/attachment.html>

From steve at pearwood.info  Sat Aug  8 07:51:26 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Aug 2009 15:51:26 +1000
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com>
Message-ID: <200908081551.26421.steve@pearwood.info>

On Fri, 7 Aug 2009 09:43:54 pm Tal Einat wrote:

> You can already do bitwise operations on bytes, 

Are you sure about that? I tried looking it up in the docs, but 
python.org is down at the moment, so excuse me if I've missed something 
obvious.

In Python 3.0:

>>> b'ABCD' & 255
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for &: 'bytes' and 'int'
>>>
>>> b'ABCD' & b'\xff'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for &: 'bytes' and 'bytes'


Have I missed something?



> As I see it, the major underlying problem here is that 
> byte-arrays are immutable, since what you really want is to be able
> to change a single byte of the array in-place. 

I'm not speaking for the original poster, but for myself, not 
necessarily. While it would be nice to be able to flip bits directly, 
I'd be happy with bitwise operators to return new instances, as they 
currently do for ints. Something like:

# not actual code
>>> bb = bytes("ABCD") 
>>> bb &= b"\xff\0" 
>>> bb
b"C\0" 

This could work for bit-flipping operations, although I'd welcome an 
easier to use API:

# still not actual code
>>> bb = bytes("ABCD")
>>> bb |= (2**5)*256  # set the 5th bit of the 2nd byte
>>> bb
b"ABcD"



-- 
Steven D'Aprano


From steve at pearwood.info  Sat Aug  8 07:54:36 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 8 Aug 2009 15:54:36 +1000
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
Message-ID: <200908081554.36358.steve@pearwood.info>

On Fri, 7 Aug 2009 11:19:20 pm Mark Dickinson wrote:

> > Also conceivable is using the shift operators >> and << on bytes,
> > but I personally would use that less often, and the result of such
> > an operation is ambiguous due to endianness.
>
> Agreed.  To make sense of the shift operators you effectively have
> to give 'position' interpretations for the individual bits, and
> there's no single obvious way of doing this;  for the plain bitwise
> operations this isn't necessary.

To me, the single obvious meaning of left- and right-shift is to shift 
to the left and the right :)

E.g. 

b"abcd" >> 8
=> "abc"

b"abcd" << 8 
=> "abcd\0"

which would have the benefit of matching what ints already do:

>>> [hex(ord(c)) for c in "ABCD"]
['0x41', '0x42', '0x43', '0x44']
>>> n = 0x41424344
>>> hex(n >> 8)
'0x414243'
>>> hex(n << 8)
'0x4142434400'


I'm not sure what other "obvious" meanings you could give them. Have I 
missed something?



-- 
Steven D'Aprano


From bruce at leapyear.org  Sat Aug  8 08:55:43 2009
From: bruce at leapyear.org (Bruce Leban)
Date: Fri, 7 Aug 2009 23:55:43 -0700
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <4A7CCB94.8020408@canterbury.ac.nz>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com> 
	<87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz>
Message-ID: <cf5b87740908072355x767e9f49jbfc0b08300456443@mail.gmail.com>

I notice you said "args" not "arg". How would you parse foo(x, y -> x + y)?
Even if the compiler can parse it, how about the human reading the code?

If lambda wasn't already there, I could see using (args) -> expr syntax but
I think there shouldn't be two ways to do something so simple.

If you really want a shorter syntax, the obvious choice is foo(?x, y: x +
y).

--- Bruce


On Fri, Aug 7, 2009 at 5:49 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> My suggestion for a lightweight lambda syntax is
>
>  args -> expr
>
> Examples:
>
>  map(x -> x + 5, something)
>
>  d = defauldict(->[])
>
> --
> Greg
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090807/e78990dd/attachment.html>

From masklinn at masklinn.net  Sat Aug  8 10:18:39 2009
From: masklinn at masklinn.net (Masklinn)
Date: Sat, 8 Aug 2009 10:18:39 +0200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <cf5b87740908072355x767e9f49jbfc0b08300456443@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<87d477lnuk.fsf@benfinney.id.au>
	<4A7CCB94.8020408@canterbury.ac.nz>
	<cf5b87740908072355x767e9f49jbfc0b08300456443@mail.gmail.com>
Message-ID: <9129F740-971E-4188-A946-A80A38313B31@masklinn.net>

On 8 Aug 2009, at 08:55 , Bruce Leban wrote:
> If you really want a shorter syntax, the obvious choice is foo(?x,  
> y: x +
> y).
>
> --- Bruce
+1

From greg.ewing at canterbury.ac.nz  Sat Aug  8 13:00:15 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 08 Aug 2009 23:00:15 +1200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <cf5b87740908072355x767e9f49jbfc0b08300456443@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz>
	<cf5b87740908072355x767e9f49jbfc0b08300456443@mail.gmail.com>
Message-ID: <4A7D5ABF.7020205@canterbury.ac.nz>

Bruce Leban wrote:
> I notice you said "args" not "arg". How would you parse foo(x, y -> x + 
> y)?

Function foo has 2 arguments there.

The other way would be

    foo((x, y) -> x + y)

-- 
Greg


From zuo at chopin.edu.pl  Sat Aug  8 13:19:07 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 08 Aug 2009 13:19:07 +0200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <4A7CCB94.8020408@canterbury.ac.nz>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz>
Message-ID: <op.uyccd5wbfvx12t@jerzozwiesz.home.aster.pl>

08-08-2009 Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> My suggestion for a lightweight lambda syntax is
>
>    args -> expr
>
> Examples:
>
>    map(x -> x + 5, something)
>
>    d = defauldict(->[])

-1
The idea would be nice, but "->" has already another meaning (in a
different context -- function annotations -- but it's not a good
idea to use the same "operator" for two completely different things,
both connected with functions).

*j


From zuo at chopin.edu.pl  Sat Aug  8 13:22:58 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sat, 08 Aug 2009 13:22:58 +0200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
Message-ID: <op.uycckkmofvx12t@jerzozwiesz.home.aster.pl>

> I was thinking about a good syntax for implicit lambdas for a while
> and today I had this idea: make ``_:`` a shortcut for ``lambda
> _=None:``

-1

I personally feel it as ugly and not readable (visually) at first
sight...

And, as others noted, '_' is already being used for a few completely
different things.

*j


From solipsis at pitrou.net  Sat Aug  8 16:51:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 8 Aug 2009 14:51:28 +0000 (UTC)
Subject: [Python-ideas] bitwise operations on bytes and buffer objects
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
Message-ID: <loom.20090808T144913-659@post.gmane.org>

Mark Dickinson <dickinsm at ...> writes:
> 
> +1 from me.  I'd say that it makes just as much sense to do bit
> operations on bytes or bytearrays as it does on integers.

We should also make it work on memoryviews.

Regards

Antoine.




From guido at python.org  Sat Aug  8 23:31:32 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Aug 2009 14:31:32 -0700
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <200908081554.36358.steve@pearwood.info>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> 
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> 
	<200908081554.36358.steve@pearwood.info>
Message-ID: <ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>

On Fri, Aug 7, 2009 at 10:54 PM, Steven D'Aprano<steve at pearwood.info> wrote:
> On Fri, 7 Aug 2009 11:19:20 pm Mark Dickinson wrote:
>
>> > Also conceivable is using the shift operators >> and << on bytes,
>> > but I personally would use that less often, and the result of such
>> > an operation is ambiguous due to endianness.
>>
>> Agreed. ?To make sense of the shift operators you effectively have
>> to give 'position' interpretations for the individual bits, and
>> there's no single obvious way of doing this; ?for the plain bitwise
>> operations this isn't necessary.
>
> To me, the single obvious meaning of left- and right-shift is to shift
> to the left and the right :)
>
> E.g.
>
> b"abcd" >> 8
> => "abc"
>
> b"abcd" << 8
> => "abcd\0"
>
> which would have the benefit of matching what ints already do:
>
>>>> [hex(ord(c)) for c in "ABCD"]
> ['0x41', '0x42', '0x43', '0x44']
>>>> n = 0x41424344
>>>> hex(n >> 8)
> '0x414243'
>>>> hex(n << 8)
> '0x4142434400'
>
>
> I'm not sure what other "obvious" meanings you could give them. Have I
> missed something?

Yes, byte order. It is not at all "obvious" whether the lowest-order
byte is on the left or on the right. Your interpretation is
big-endian. But mathematically speaking, a little-endian
interpretation is somewhat easier, because the value in byte number i
corresponds to that value multiplied by 256**i. Another way to look at
is, is b'abc' supposed to be equal to b'\0abc' (big-endian) or
b'abc\0' (little-endian) ? I find ignoring trailing nulls more logical
than ignoring leading nulls, since the indexes of the significant
digits are the same in the little-endian case.

In the grander scheme of things, I worry that interpreting byte
strings as integers and implementing bitwise operators on them is
going to cause more confusion and isn't generally useful enough to
warrant the extra code. I'd be okay with a standard API to transform a
byte array into an integer and vice versa -- there you can be explicit
about byte order and what to do about negative numbers. I can't
remember right now if we already have such an API for arbitrary sizes
-- the struct module only handles sizes 2, 4 and 8. I can hack it by
going via a hex representation:

  i = 10**100
  b = bytes.fromhex(hex(i)[2:])
  import binascii
  j = int(binascii.hexlify(b), 16)
  assert j == i

but this is a pretty gross hack. Still, most likely faster than
writing out a loop in Python.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From g.brandl at gmx.net  Sun Aug  9 00:09:42 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 09 Aug 2009 00:09:42 +0200
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
Message-ID: <h5kt5n$hub$1@ger.gmane.org>

Guido van Rossum schrieb:

> In the grander scheme of things, I worry that interpreting byte
> strings as integers and implementing bitwise operators on them is
> going to cause more confusion and isn't generally useful enough to
> warrant the extra code.

What about operations that don't require the bytes to be interpreted
as anything else?

The OP proposed

bytes_a | bytes_b

to mean

bytes(b_a | b_b for (b_a, b_b) in zip(bytes_a, bytes_b))

except that (probably) an equal length would have to be asserted.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From guido at python.org  Sun Aug  9 00:27:46 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Aug 2009 15:27:46 -0700
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <h5kt5n$hub$1@ger.gmane.org>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> 
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> 
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com> 
	<h5kt5n$hub$1@ger.gmane.org>
Message-ID: <ca471dc20908081527p17327a65icbcf06d8e4295ba5@mail.gmail.com>

I think it's a very rarely used feature that is more likely to baffle
the casual reader. The question of what to do with byte strings of
unequal length is just the first issue that crops up. The (still
legitimate) question why we would support | and & but not << and >> is
another.

It's a slippery slope...

--Guido

On Sat, Aug 8, 2009 at 3:09 PM, Georg Brandl<g.brandl at gmx.net> wrote:
> Guido van Rossum schrieb:
>
>> In the grander scheme of things, I worry that interpreting byte
>> strings as integers and implementing bitwise operators on them is
>> going to cause more confusion and isn't generally useful enough to
>> warrant the extra code.
>
> What about operations that don't require the bytes to be interpreted
> as anything else?
>
> The OP proposed
>
> bytes_a | bytes_b
>
> to mean
>
> bytes(b_a | b_b for (b_a, b_b) in zip(bytes_a, bytes_b))
>
> except that (probably) an equal length would have to be asserted.
>
> Georg
>
> --
> Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
> Four shall be the number of spaces thou shalt indent, and the number of thy
> indenting shall be four. Eight shalt thou not indent, nor either indent thou
> two, excepting that thou then proceed to four. Tabs are right out.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From greg.ewing at canterbury.ac.nz  Sun Aug  9 00:41:55 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 09 Aug 2009 10:41:55 +1200
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <ca471dc20908081527p17327a65icbcf06d8e4295ba5@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<h5kt5n$hub$1@ger.gmane.org>
	<ca471dc20908081527p17327a65icbcf06d8e4295ba5@mail.gmail.com>
Message-ID: <4A7DFF33.8050003@canterbury.ac.nz>

Guido van Rossum wrote:
> I think it's a very rarely used feature that is more likely to baffle
> the casual reader.

However there's currently no way to efficiently do
bitwise operations en masse on raw bytes without
converting to and from long integers, which doesn't
seem very satisfactory. Refusing to provide this
ability for the above reason sounds like purity
beating practicality to me.

> The question of what to do with byte strings of
> unequal length is just the first issue that crops up.

I would raise an exception.

> The (still
> legitimate) question why we would support | and & but not << and >> is
> another.

Ambiguity due to byte order seems like a good enough
reason not to implement shifts as operators. Maybe
provide methods that specify a byte order explicitly?

-- 
Greg


From mwm-keyword-python.b4bdba at mired.org  Sun Aug  9 01:22:07 2009
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Sat, 8 Aug 2009 19:22:07 -0400
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <4A7DFF33.8050003@canterbury.ac.nz>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<h5kt5n$hub$1@ger.gmane.org>
	<ca471dc20908081527p17327a65icbcf06d8e4295ba5@mail.gmail.com>
	<4A7DFF33.8050003@canterbury.ac.nz>
Message-ID: <20090808192207.31c36098@bhuda.mired.org>

On Sun, 09 Aug 2009 10:41:55 +1200
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Guido van Rossum wrote:
> > I think it's a very rarely used feature that is more likely to baffle
> > the casual reader.
> 
> However there's currently no way to efficiently do
> bitwise operations en masse on raw bytes without
> converting to and from long integers, which doesn't
> seem very satisfactory. Refusing to provide this
> ability for the above reason sounds like purity
> beating practicality to me.

It seems to me that bytes are a container type. How many builtin
operators are there that operate on every element in any kind of
container (other than queries, many of which already work on bytes)?

It seems to me that what you really want is a high-performance array
module of some flavor. Isn't that part of what NumPy provides?

       <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From facundobatista at gmail.com  Sun Aug  9 07:29:37 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Sun, 9 Aug 2009 02:29:37 -0300
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
Message-ID: <e04bdf310908082229v14beccc1u7fd4406e67128e33@mail.gmail.com>

On Sat, Aug 8, 2009 at 6:31 PM, Guido van Rossum<guido at python.org> wrote:

> In the grander scheme of things, I worry that interpreting byte
> strings as integers and implementing bitwise operators on them is
> going to cause more confusion and isn't generally useful enough to

Indeed.  But let me rephrase this.

I don't know Python, I'm just starting with Py3k, and I see a bytes
object. I don't know what a "byte string" is  (and I even feel that
the term is strange), but I understand that b"b3\xdd" is an array of
three bytes, which, of course, is an array of 24 bits (unless I know
that bytes are not always octets, but I think it's the same for this
case).

So, I think that providing bit semantics is a must for bytes objects.

I don't care (or I don't know enough to know that this exist) about
big/little endian, I just assume that this "just works" (it's is the
same case with line ending bytes: I may not know that different OSs
finish lines with different marks).

*I* (me as Facundo), don't really know enough to be able to propose
"what could work and keep surprise to a minimum", but just wanted to
present an "end user case" about this.


> warrant the extra code. I'd be okay with a standard API to transform a
> byte array into an integer and vice versa -- there you can be explicit
> about byte order and what to do about negative numbers. I can't

Yes. Bit arrays and integers suffer (suffer?) from the same issue that
Unicode. An integer, and an Unicode character, are encoded into
bits... and if you have bits, you need to know how to decode them to
get again your Unicode character, or your integer.

Maybe we could use the same names? What about something like b"....".
decode("little_endian__two's_complement") --> int?  (please, very
please, better encoding names)


Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/


From ede at mit.edu  Sun Aug  9 09:06:32 2009
From: ede at mit.edu (Eric Eisner)
Date: Sun, 9 Aug 2009 16:06:32 +0900
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
Message-ID: <42ef4ee0908090006q774923b5p17fc6ae7b1bff32b@mail.gmail.com>

> In the grander scheme of things, I worry that interpreting byte
> strings as integers and implementing bitwise operators on them is
> going to cause more confusion and isn't generally useful enough to
> warrant the extra code. I'd be okay with a standard API to transform a
> byte array into an integer and vice versa -- there you can be explicit
> about byte order and what to do about negative numbers.

For some background, I think the separation of text and data is a
powerful idea, and now that I have my nice builtin object for data
(and its mutable cousin), I want to be able to manipulate the raw data
directly. Currently, all of the bitwise manipulations live in the int
type (for what I like to think of as historical reasons). Currently,
when manipulating a byte at a time, the bytes type does the right
thing and shows the elements as ints, giving me access to these
bitwise manipulations. However there currently lacks builtin
functionality to manipulate multi-bytes chunks. In the grander scheme
of things, this is what I would like to see addressed.

The most general solution, which Guido mentions, and which I have also
been thinking about, is a builtin conversion from bytes to ints that
explicitly requires resolving ambiguities (eg endianness, negatives,
data length). I personally use those pretty gross hacks to fake the
functionality now, but they assumes big-endian, two's complement etc.

I proposed direct bitwise operators (~ & ^ |) for bytes because these
seemed to be the subset of all bitwise manipulations that were
(mostly) unambiguous for raw bytes. Even if python gains conversions
between bytes and ints, I still think these unambiguous operations
would be useful to have (especially ~, the most unambiguous among
them).

-Eric


From ilya.nikokoshev at gmail.com  Sun Aug  9 09:29:37 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Sun, 9 Aug 2009 11:29:37 +0400
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <200908081021.15580.steve@pearwood.info>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<200908081021.15580.steve@pearwood.info>
Message-ID: <d8cc6f9d0908090029r745ad072i59a28eca6b082cc2@mail.gmail.com>

Thank you and everyone else for insightful posts detailing why my
examples don't make a good argument for the syntax.

Even though my original suggestion, similarly pep 312, wouldn't break
any existing programs and would not lead to ambiguity in 'if _:', I
rescind it.

However, another reason for implicit lambdas is lazy evaluation. For
example, in another thread people discuss "... except ... if/else"
conditional statement --- one reason being control expressions
evaluate lazily. A function call passing callables currently looks
ugly and unreadable:

    lazy_cond(expr, lambda: expensive(5), lambda: factorial(10**5))

and here 6 keystrokes of 'lambda' word *do* matter.

Therefore I hope my unsuccessful proposal will encourage people to
find something that works.


On Sat, Aug 8, 2009 at 4:21 AM, Steven D'Aprano<steve at pearwood.info> wrote:
> On Fri, 7 Aug 2009 10:46:40 pm ilya wrote:
>> I was thinking about a good syntax for implicit lambdas for a while
>> and today I had this idea: make ``_:`` a shortcut for ``lambda
>> _=None:``
>
> [...]
>
>> The rationale is that you only want to get rid of lambda keyword to
>> create a *very* simple function, the one that will be called either
>> without parameters or with only one parameter. For everything more
>> complicated, you really should go and write the explicit function
>> signature using lambda.
>
> Why would you want to get rid of the lambda keyword? What's the benefit?
>
> Is this about saving twelve keystrokes?
>
> lambda _=None:
> ?versus
> _:
>
> Just how often do you want, or need, to write such a lambda? It seems to
> me that not only is it a special case you want to break the rules for,
> which goes against the Zen, but it's an incredibly rare special case.
>
> _ as an identifier already has three conventional meanings:
>
> (1) In the interactive interpreter, _ is the value of the last
> expression typed.
>
> (2) It is commonly used to mean "I don't care about this value", e.g.
>
> t = ("Fred Smith", "123 Fourth Street", "New York", "dog")
> name, _, _, pet = t
>
> (3) It is also often used in internationalization.
>
> You want to give it the extra meaning "a default parameter name for
> lambda when I can't be bothered typing even a single letter name".
>
> Because _ is already a valid identifier, this will break code that does
> this:
>
> while _:
> ? ?process()
> ? ?_ = function()
>
> if _:
> ? ?print "something"
>
>
> Not the best choice of names, but somebody, somewhere, is doing that,
> and your suggestion will break their code.
>
>
> Looking at the three examples you gave:
>
> map( _: _ + 5, some_list)
> register_callback( _: True)
> def apply_transform(..., transform = _:_, ... ):
>
> In the first case, I wouldn't use the short-cut form even if it were
> available. I'd write a lambda that used a more meaningful name. In this
> case, I'm expecting an int, so I would use n, or a float, so I'd use x.
> I'd also avoid setting the pointless default:
>
> map(lambda x: x+5, some_list)
> ?vs
> map(_: _+5, some_list)
>
> Since your suggestion doesn't do precisely what I want, the only reason
> I would have for using your construct is to save seven keystrokes.
> Encouraging laziness on the behalf of the programmer is not a good
> reason for special-casing rare cases.
>
> Second case: register_callback( _: True)
>
> I assume you're implying that the callback function must take a single
> argument. In this example, using _ as the parameter name to the lambda
> makes sense, because it is a "don't care" argument. But if the callback
> function is documented as always being given a single argument, I would
> want to know if it was being called without any arguments, so the
> default value of None is inappropriate and I would avoid using it.
>
> Third case: def apply_transform(..., transform = _:_, ... ):
>
> I don't think I'd write a function called apply_transform() which made
> the transformation function optional, let alone buried deep in the
> middle of a whole lot of extra parameters. (I presume that's what
> the "..."s are meant to imply.) But putting that aside, I see your
> intention: a default do-nothing function which appears in a very long
> parameter list. The problem is that instead of trying to shrink the
> default value so you can fit all the parameters on a single line, you
> should make such a complicated function signature more readable by
> spreading it out:
>
> def apply_transform(
> ? ?obj,
> ? ?start, end, ? ? ?# start (inc) and end (exc) positions to apply
> ? ?another_arg, ? ? # does something very important I'm sure
> ? ?x=0, y=1, z=2, ? # more very important arguments
> ? ?transform=( ? ? ?# default null transformation
> ? ? ? ? ? ? ?lambda obj=None: obj),
> ? ?frotz=False, ? ? # if true, frotz the hymangirator with spangule
> ? ?hymangirator=None,
> ? ?spangule=None,
> ? ?magic=12345, ? ? # this needs no documentation
> ? ?namespace={},
> ? ?flibbertigibbet=None,
> ? ?_private_magic=[] ?# the caller shouldn't supply this
> ? ?):
>
> (Even better is to avoid such complicated function signatures, but
> sometimes that's not an option.)
>
> So again I'd be very unlikely to use your suggested construct except out
> of simple can't-be-bothered-to-type-a-dozen-letters laziness. Pandering
> to that sort of laziness is probably not a good thing.
>
> Fundamentally, this suggestion doesn't add expressability to the
> language, or power. Laziness on it's own is not a good reason for
> special casing rare cases. If it was a very common case, then *perhaps*
> you would have an argument for special casing needless verbiage:
> conciseness (up to a point) is a virtue in a language. That's partly
> why we have lambdas in the first place, so we can write this:
>
> reduce(lambda a,b: (a+b)/2.0, somelist)
>
> instead of this:
>
> def average(a, b):
> ? ?return (a+b)/2.0
> reduce(average, somelist)
>
> But this isn't a common case, it's a rare case, and the case you're
> hoping to replace is pretty concise already.
>
>
>
> --
> Steven D'Aprano
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From gerald.britton at gmail.com  Sun Aug  9 13:03:43 2009
From: gerald.britton at gmail.com (Gerald Britton)
Date: Sun, 9 Aug 2009 07:03:43 -0400
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <d8cc6f9d0908090029r745ad072i59a28eca6b082cc2@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com> 
	<200908081021.15580.steve@pearwood.info>
	<d8cc6f9d0908090029r745ad072i59a28eca6b082cc2@mail.gmail.com>
Message-ID: <5d1a32000908090403h53d1d25ai5b7e72e06cb591fa@mail.gmail.com>

Your proposal certainly would lead to ambiguity in reading:
_ = myfunc
if _:_(_:_, ...)

The "if _:_" means:

   if '_' evaluates to boolean true, call it

The second _:_ uses your new lambda construct.  Could a compiler parse it?
Probably.  Can a human understand it? Maybe, with difficulty, but it would
surely lead to hard-to-find errors.

On Sun, Aug 9, 2009 at 3:29 AM, ilya <ilya.nikokoshev at gmail.com> wrote:

> Thank you and everyone else for insightful posts detailing why my
> examples don't make a good argument for the syntax.
>
> Even though my original suggestion, similarly pep 312, wouldn't break
> any existing programs and would not lead to ambiguity in 'if _:', I
> rescind it.
>
> However, another reason for implicit lambdas is lazy evaluation. For
> example, in another thread people discuss "... except ... if/else"
> conditional statement --- one reason being control expressions
> evaluate lazily. A function call passing callables currently looks
> ugly and unreadable:
>
>    lazy_cond(expr, lambda: expensive(5), lambda: factorial(10**5))
>
> and here 6 keystrokes of 'lambda' word *do* matter.
>
> Therefore I hope my unsuccessful proposal will encourage people to
> find something that works.
>
>
> On Sat, Aug 8, 2009 at 4:21 AM, Steven D'Aprano<steve at pearwood.info>
> wrote:
> > On Fri, 7 Aug 2009 10:46:40 pm ilya wrote:
> >> I was thinking about a good syntax for implicit lambdas for a while
> >> and today I had this idea: make ``_:`` a shortcut for ``lambda
> >> _=None:``
> >
> > [...]
> >
> >> The rationale is that you only want to get rid of lambda keyword to
> >> create a *very* simple function, the one that will be called either
> >> without parameters or with only one parameter. For everything more
> >> complicated, you really should go and write the explicit function
> >> signature using lambda.
> >
> > Why would you want to get rid of the lambda keyword? What's the benefit?
> >
> > Is this about saving twelve keystrokes?
> >
> > lambda _=None:
> >  versus
> > _:
> >
> > Just how often do you want, or need, to write such a lambda? It seems to
> > me that not only is it a special case you want to break the rules for,
> > which goes against the Zen, but it's an incredibly rare special case.
> >
> > _ as an identifier already has three conventional meanings:
> >
> > (1) In the interactive interpreter, _ is the value of the last
> > expression typed.
> >
> > (2) It is commonly used to mean "I don't care about this value", e.g.
> >
> > t = ("Fred Smith", "123 Fourth Street", "New York", "dog")
> > name, _, _, pet = t
> >
> > (3) It is also often used in internationalization.
> >
> > You want to give it the extra meaning "a default parameter name for
> > lambda when I can't be bothered typing even a single letter name".
> >
> > Because _ is already a valid identifier, this will break code that does
> > this:
> >
> > while _:
> >    process()
> >    _ = function()
> >
> > if _:
> >    print "something"
> >
> >
> > Not the best choice of names, but somebody, somewhere, is doing that,
> > and your suggestion will break their code.
> >
> >
> > Looking at the three examples you gave:
> >
> > map( _: _ + 5, some_list)
> > register_callback( _: True)
> > def apply_transform(..., transform = _:_, ... ):
> >
> > In the first case, I wouldn't use the short-cut form even if it were
> > available. I'd write a lambda that used a more meaningful name. In this
> > case, I'm expecting an int, so I would use n, or a float, so I'd use x.
> > I'd also avoid setting the pointless default:
> >
> > map(lambda x: x+5, some_list)
> >  vs
> > map(_: _+5, some_list)
> >
> > Since your suggestion doesn't do precisely what I want, the only reason
> > I would have for using your construct is to save seven keystrokes.
> > Encouraging laziness on the behalf of the programmer is not a good
> > reason for special-casing rare cases.
> >
> > Second case: register_callback( _: True)
> >
> > I assume you're implying that the callback function must take a single
> > argument. In this example, using _ as the parameter name to the lambda
> > makes sense, because it is a "don't care" argument. But if the callback
> > function is documented as always being given a single argument, I would
> > want to know if it was being called without any arguments, so the
> > default value of None is inappropriate and I would avoid using it.
> >
> > Third case: def apply_transform(..., transform = _:_, ... ):
> >
> > I don't think I'd write a function called apply_transform() which made
> > the transformation function optional, let alone buried deep in the
> > middle of a whole lot of extra parameters. (I presume that's what
> > the "..."s are meant to imply.) But putting that aside, I see your
> > intention: a default do-nothing function which appears in a very long
> > parameter list. The problem is that instead of trying to shrink the
> > default value so you can fit all the parameters on a single line, you
> > should make such a complicated function signature more readable by
> > spreading it out:
> >
> > def apply_transform(
> >    obj,
> >    start, end,      # start (inc) and end (exc) positions to apply
> >    another_arg,     # does something very important I'm sure
> >    x=0, y=1, z=2,   # more very important arguments
> >    transform=(      # default null transformation
> >              lambda obj=None: obj),
> >    frotz=False,     # if true, frotz the hymangirator with spangule
> >    hymangirator=None,
> >    spangule=None,
> >    magic=12345,     # this needs no documentation
> >    namespace={},
> >    flibbertigibbet=None,
> >    _private_magic=[]  # the caller shouldn't supply this
> >    ):
> >
> > (Even better is to avoid such complicated function signatures, but
> > sometimes that's not an option.)
> >
> > So again I'd be very unlikely to use your suggested construct except out
> > of simple can't-be-bothered-to-type-a-dozen-letters laziness. Pandering
> > to that sort of laziness is probably not a good thing.
> >
> > Fundamentally, this suggestion doesn't add expressability to the
> > language, or power. Laziness on it's own is not a good reason for
> > special casing rare cases. If it was a very common case, then *perhaps*
> > you would have an argument for special casing needless verbiage:
> > conciseness (up to a point) is a virtue in a language. That's partly
> > why we have lambdas in the first place, so we can write this:
> >
> > reduce(lambda a,b: (a+b)/2.0, somelist)
> >
> > instead of this:
> >
> > def average(a, b):
> >    return (a+b)/2.0
> > reduce(average, somelist)
> >
> > But this isn't a common case, it's a rare case, and the case you're
> > hoping to replace is pretty concise already.
> >
> >
> >
> > --
> > Steven D'Aprano
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > http://mail.python.org/mailman/listinfo/python-ideas
> >
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
Gerald Britton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090809/99b02a02/attachment.html>

From masklinn at masklinn.net  Sun Aug  9 16:40:15 2009
From: masklinn at masklinn.net (Masklinn)
Date: Sun, 9 Aug 2009 16:40:15 +0200
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <5d1a32000908090403h53d1d25ai5b7e72e06cb591fa@mail.gmail.com>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<200908081021.15580.steve@pearwood.info>
	<d8cc6f9d0908090029r745ad072i59a28eca6b082cc2@mail.gmail.com>
	<5d1a32000908090403h53d1d25ai5b7e72e06cb591fa@mail.gmail.com>
Message-ID: <9226C8D2-0C02-4D63-BA19-66A210D005A9@masklinn.net>

On 9 Aug 2009, at 13:03 , Gerald Britton wrote:
> Your proposal certainly would lead to ambiguity in reading:
> _ = myfunc
> if _:_(_:_, ...)
>
> The "if _:_" means:
>
>   if '_' evaluates to boolean true, call it
>
> The second _:_ uses your new lambda construct.  Could a compiler  
> parse it?
> Probably.  Can a human understand it? Maybe, with difficulty, but it  
> would
> surely lead to hard-to-find errors.

And from the point of view of someone who'd really like a "better  
lambda", the "implicit lambda" idea is pretty much worthless. As  
others have said, it saves a few keystrokes and that's pretty much it.

If work is done towards a better/more useable lambda, it should at  
least encompass full-blown anonymous expressions, not limit itself to  
the current restricted lambda.


From guido at python.org  Sun Aug  9 22:57:26 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 9 Aug 2009 13:57:26 -0700
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <e04bdf310908082229v14beccc1u7fd4406e67128e33@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> 
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> 
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com> 
	<e04bdf310908082229v14beccc1u7fd4406e67128e33@mail.gmail.com>
Message-ID: <ca471dc20908091357n693e7221qd22fb8955fb1bfbb@mail.gmail.com>

On Sat, Aug 8, 2009 at 10:29 PM, Facundo
Batista<facundobatista at gmail.com> wrote:
> On Sat, Aug 8, 2009 at 6:31 PM, Guido van Rossum<guido at python.org> wrote:
>
>> In the grander scheme of things, I worry that interpreting byte
>> strings as integers and implementing bitwise operators on them is
>> going to cause more confusion and isn't generally useful enough to
>
> Indeed. ?But let me rephrase this.
>
> I don't know Python, I'm just starting with Py3k, and I see a bytes
> object. I don't know what a "byte string" is ?(and I even feel that
> the term is strange), but I understand that b"b3\xdd" is an array of
> three bytes, which, of course, is an array of 24 bits (unless I know
> that bytes are not always octets, but I think it's the same for this
> case).
>
> So, I think that providing bit semantics is a must for bytes objects.
>
> I don't care (or I don't know enough to know that this exist) about
> big/little endian, I just assume that this "just works" (it's is the
> same case with line ending bytes: I may not know that different OSs
> finish lines with different marks).

Sounds like a non-sequitur if anything.

>From the perspective of "I don't know Python" you cannot expect to
draw valid conclusions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From python at mrabarnett.plus.com  Sun Aug  9 23:32:14 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 09 Aug 2009 22:32:14 +0100
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <ca471dc20908091357n693e7221qd22fb8955fb1bfbb@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<e04bdf310908082229v14beccc1u7fd4406e67128e33@mail.gmail.com>
	<ca471dc20908091357n693e7221qd22fb8955fb1bfbb@mail.gmail.com>
Message-ID: <4A7F405E.90409@mrabarnett.plus.com>

Guido van Rossum wrote:
> On Sat, Aug 8, 2009 at 10:29 PM, Facundo
> Batista<facundobatista at gmail.com> wrote:
>> On Sat, Aug 8, 2009 at 6:31 PM, Guido van Rossum<guido at python.org> wrote:
>>
>>> In the grander scheme of things, I worry that interpreting byte
>>> strings as integers and implementing bitwise operators on them is
>>> going to cause more confusion and isn't generally useful enough to
>> Indeed.  But let me rephrase this.
>>
>> I don't know Python, I'm just starting with Py3k, and I see a bytes
>> object. I don't know what a "byte string" is  (and I even feel that
>> the term is strange), but I understand that b"b3\xdd" is an array of
>> three bytes, which, of course, is an array of 24 bits (unless I know
>> that bytes are not always octets, but I think it's the same for this
>> case).
>>
>> So, I think that providing bit semantics is a must for bytes objects.
>>
>> I don't care (or I don't know enough to know that this exist) about
>> big/little endian, I just assume that this "just works" (it's is the
>> same case with line ending bytes: I may not know that different OSs
>> finish lines with different marks).
> 
> Sounds like a non-sequitur if anything.
> 
>>From the perspective of "I don't know Python" you cannot expect to
> draw valid conclusions.
> 
I think that bitstrings would be better handled by a distinct bitstring
class.


From dickinsm at gmail.com  Mon Aug 10 10:05:30 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Aug 2009 09:05:30 +0100
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
Message-ID: <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>

On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum<guido at python.org> wrote:
> [...] I'd be okay with a standard API to transform a
> byte array into an integer and vice versa -- there you can be explicit
> about byte order and what to do about negative numbers. [...]

That would also be a welcome addition.  It's been requested on
bugs.python.org at least a couple of times[1][2], and the C code
to do the conversions already exists (_PyLong_{As,From}_ByteArray
in longobject.c), so it wouldn't be too much work to implement.

The main problem would be deciding exactly what the API should be
and where to put it.

Mark

[1] http://bugs.python.org/issue1023290
[2] http://bugs.python.org/issue923643


From dickinsm at gmail.com  Mon Aug 10 11:23:13 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Aug 2009 10:23:13 +0100
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
Message-ID: <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com>

On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum<guido at python.org> wrote:
> -- the struct module only handles sizes 2, 4 and 8. I can hack it by
> going via a hex representation:
>
> ?i = 10**100
> ?b = bytes.fromhex(hex(i)[2:])
> ?import binascii
> ?j = int(binascii.hexlify(b), 16)
> ?assert j == i
>
> but this is a pretty gross hack.

The first part also doesn't work if hex(i) has odd length.
[py3k]:

>>> bytes.fromhex(hex(10**99)[2:])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: non-hexadecimal number found in fromhex() arg at position 82

I think the fact that it's non-trivial to get this right first
time is further evidence that it would be useful to have
built-in int <-> bytes conversions somewhere.

Mark


From ncoghlan at gmail.com  Mon Aug 10 12:30:38 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 10 Aug 2009 20:30:38 +1000
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>	<200908081554.36358.steve@pearwood.info>	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
Message-ID: <4A7FF6CE.7070903@gmail.com>

Mark Dickinson wrote:
> On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum<guido at python.org> wrote:
>> [...] I'd be okay with a standard API to transform a
>> byte array into an integer and vice versa -- there you can be explicit
>> about byte order and what to do about negative numbers. [...]
> 
> That would also be a welcome addition.  It's been requested on
> bugs.python.org at least a couple of times[1][2], and the C code
> to do the conversions already exists (_PyLong_{As,From}_ByteArray
> in longobject.c), so it wouldn't be too much work to implement.
> 
> The main problem would be deciding exactly what the API should be
> and where to put it.

My suggestion would be to provide the relevant constructors as class
methods on int, bytes and bytearray:

bytes.from_int
bytearray.from_int
int.from_bytes

Alternatively, the int.from_bytes classmethod could be replaced with a
"to_int" instance method on bytes and bytearray.

The method signatures would need to closely resemble the C API. In
particular, for the conversion from int to bytes being able to state a
desired size would both allow detection of cases where the value is too
large as well as proper padding of the two's complement sign bit.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ede at mit.edu  Mon Aug 10 13:02:08 2009
From: ede at mit.edu (Eric Eisner)
Date: Mon, 10 Aug 2009 20:02:08 +0900
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <4A7FF6CE.7070903@gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
Message-ID: <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>

On Mon, Aug 10, 2009 at 19:30, Nick Coghlan<ncoghlan at gmail.com> wrote:
> Mark Dickinson wrote:
>> The main problem would be deciding exactly what the API should be
>> and where to put it.
>
> My suggestion would be to provide the relevant constructors as class
> methods on int, bytes and bytearray:
>
> bytes.from_int
> bytearray.from_int
> int.from_bytes
>
> Alternatively, the int.from_bytes classmethod could be replaced with a
> "to_int" instance method on bytes and bytearray.
>
> The method signatures would need to closely resemble the C API. In
> particular, for the conversion from int to bytes being able to state a
> desired size would both allow detection of cases where the value is too
> large as well as proper padding of the two's complement sign bit.

For completeness, any function converting from int to bytes needs to
accept the arguments size, endianness, and two's complement handling.
By default, size and two's complement could be inferred from the int's
value.

Any function converting from bytes to int needs to accept the
arguments endianness and two's complement. The function cannot really
make a reasonable assumption about either.

Having this much to specify in bytes.from_int or bytes.to_int seems a
little overwhelming (to me anyway). It is already more complicated
than the analogous bytes.decode for strings.

What about a putting this in module support, and putting each of these
options in its own named method? Some dummy examples that may need
better names:
base256.encode_le(0x123456, size=5) # returns b'\x56\x34\x12\x00\x00'
base256.decode_le_positive(b'\xff\xce') # returns 0xceff

-Eric


From ncoghlan at gmail.com  Mon Aug 10 15:13:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 10 Aug 2009 23:13:48 +1000
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>	
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>	
	<200908081554.36358.steve@pearwood.info>	
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>	
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>	
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
Message-ID: <4A801D0C.2000502@gmail.com>

Eric Eisner wrote:
> For completeness, any function converting from int to bytes needs to
> accept the arguments size, endianness, and two's complement handling.
> By default, size and two's complement could be inferred from the int's
> value.

Keep in mind that Mark's suggestion isn't to completely reinvent the
wheel - it is more about exposing an existing internal C API to Python
code. Since that C API is provided by the int object, we should also
consider the option of providing it in Python purely as methods of that
object.

Bytes to int:

  bytes pointer: pointer to first byte to be converted
  size: number of bytes to be converted
  little_endian: flag indicating MSB is at offset 0
  signed: flag indicating whether or not the value is signed

For a Python version of this conversion, the size argument is
unnecessary, reducing the required parameters to a
bytes/bytearray/memoryview reference, an endianness marker and a
'signed' flag (to indicate whether the buffer contains an unsigned value
or a signed two's complement value).

One slight quirk of the C API that probably shouldn't be replicated is a
size of 0 translating to an integer result of zero. For the Python API,
passing in an empty byte sequence should trigger a ValueError.

In the C API, int to bytes takes the same parameters as the bytes to int
conversion, but the meaning is slightly different.

Int to bytes:

  bytes pointer: pointer to first byte to be converted
  size: number of bytes in result
  little_endian: flag indicating to write MSB to offset 0
  signed: flag indicating negative value can be converted

In this case, the size matters even for a Python version as an
OverflowError will be raised if the integer won't fit in the specified
size and sufficient sign bits are added to pad the result out to the
requested size otherwise.

That suggests to me the following signatures for the conversion
functions (regardless of the names they might be given):

int.from_bytes(data, *, little_endian=None, signed=True)
  little_endian would become a three-valued parameter for the Python
version:  None = native; False = little-endian; True = big-endian. The
three valued parameter is necessary since Python code can't get at the
"IS_LITTLE_ENDIAN" macro that the PyLong code uses to determine the
endianness of the system when calling the C API functions.
  signed would just be an ordinary boolean flag

int.to_bytes(data, *, size=0, little_endian=None, signed=True)
  A size <= 0 would mean to produce as many bytes as are needed to
represent the integer and no more. Otherwise it would represent the
maximum number of bytes allowed in the response (raising OverflowError
if the value won't fit).
  little_endian and signed would be interpreted as for the conversion
from bytes to an integer

Sure, these could be moved into a module of their own, but I'm not sure
what would be gained by doing so.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Mon Aug 10 15:38:36 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 10 Aug 2009 13:38:36 +0000 (UTC)
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>	
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>	
	<200908081554.36358.steve@pearwood.info>	
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>	
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>	
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
Message-ID: <loom.20090810T133442-435@post.gmane.org>

Nick Coghlan <ncoghlan at ...> writes:
> 
> int.to_bytes(data, *, size=0, little_endian=None, signed=True)

It would be better IMO if this method belonged to the target type, so that you
can write `bytearray.from_int()` without going through an intermediate bytes
object.

What if `data` is equal to 0? Does it return an empty bytes object? It doesn't
look dangerous to do so, so I'd say go for it.




From ncoghlan at gmail.com  Mon Aug 10 15:53:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 10 Aug 2009 23:53:53 +1000
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <loom.20090810T133442-435@post.gmane.org>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>		<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>		<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>		<200908081554.36358.steve@pearwood.info>		<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>		<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>		<4A7FF6CE.7070903@gmail.com>	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>	<4A801D0C.2000502@gmail.com>
	<loom.20090810T133442-435@post.gmane.org>
Message-ID: <4A802671.9040002@gmail.com>

Antoine Pitrou wrote:
> Nick Coghlan <ncoghlan at ...> writes:
>> int.to_bytes(data, *, size=0, little_endian=None, signed=True)
> 
> It would be better IMO if this method belonged to the target type, so that you
> can write `bytearray.from_int()` without going through an intermediate bytes
> object.
> 
> What if `data` is equal to 0? Does it return an empty bytes object? It doesn't
> look dangerous to do so, so I'd say go for it.

Actually, that's just a typo after copying the from_bytes signature line
when writing my message. int.to_bytes would just be a normal instance
method. If the value was 0, then the result would be a single byte with
the value zero (or all zeroes if a size was specified).

As far as bytearray goes, my original post did suggest that approach
(i.e. class methods on the target types), but that then gets tedious as
every new type that wants to support conversion from an integer needs to
either go through one of the existing types or else have it's own C
extension call the underlying C API directly.

Something analagous to the file.readinto() method seems like a more
appropriate solution for populating the mutable types.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From rrr at ronadam.com  Mon Aug 10 19:16:18 2009
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 10 Aug 2009 12:16:18 -0500
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>	<200908081554.36358.steve@pearwood.info>	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com>
Message-ID: <4A8055E2.2040001@ronadam.com>



Mark Dickinson wrote:
> On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum<guido at python.org> wrote:
>> -- the struct module only handles sizes 2, 4 and 8. I can hack it by
>> going via a hex representation:
>>
>>  i = 10**100
>>  b = bytes.fromhex(hex(i)[2:])
>>  import binascii
>>  j = int(binascii.hexlify(b), 16)
>>  assert j == i
>>
>> but this is a pretty gross hack.
> 
> The first part also doesn't work if hex(i) has odd length.
> [py3k]:
> 
>>>> bytes.fromhex(hex(10**99)[2:])
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: non-hexadecimal number found in fromhex() arg at position 82
> 
> I think the fact that it's non-trivial to get this right first
> time is further evidence that it would be useful to have
> built-in int <-> bytes conversions somewhere.

Are there going to possibly be other conversions to bytes and back? 
(float, string, struct, ...)

It seems to me the type conversion to and from bytes should be on the 
encoded non-byte type, and other types including user created ones could 
follow that pattern. That may allow bytes to work with any type that has 
the required special methods to do the conversions.

Then most of the methods on bytes would be for manipulating bytes in 
various ways.


The constructor for the int type already does base and string conversions, 
extending it to bytes seems like it would be natural.

    int(bytes)            # just like int(string)
    bytes = bytes(int)    # calls int.__to_bytes__() to do the actual work.


Ron




















From guido at python.org  Mon Aug 10 19:36:26 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Aug 2009 10:36:26 -0700
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <4A8055E2.2040001@ronadam.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> 
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> 
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com> 
	<5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> 
	<4A8055E2.2040001@ronadam.com>
Message-ID: <ca471dc20908101036s69f59032q4daf6f7dd8cf9647@mail.gmail.com>

The struct module already handles all those -- long ints are pretty
much the only common type that it doesn't cover, because it only deals
with fixed-length values.

On Mon, Aug 10, 2009 at 10:16 AM, Ron Adam<rrr at ronadam.com> wrote:
>
>
> Mark Dickinson wrote:
>>
>> On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum<guido at python.org> wrote:
>>>
>>> -- the struct module only handles sizes 2, 4 and 8. I can hack it by
>>> going via a hex representation:
>>>
>>> ?i = 10**100
>>> ?b = bytes.fromhex(hex(i)[2:])
>>> ?import binascii
>>> ?j = int(binascii.hexlify(b), 16)
>>> ?assert j == i
>>>
>>> but this is a pretty gross hack.
>>
>> The first part also doesn't work if hex(i) has odd length.
>> [py3k]:
>>
>>>>> bytes.fromhex(hex(10**99)[2:])
>>
>> Traceback (most recent call last):
>> ?File "<stdin>", line 1, in <module>
>> ValueError: non-hexadecimal number found in fromhex() arg at position 82
>>
>> I think the fact that it's non-trivial to get this right first
>> time is further evidence that it would be useful to have
>> built-in int <-> bytes conversions somewhere.
>
> Are there going to possibly be other conversions to bytes and back? (float,
> string, struct, ...)
>
> It seems to me the type conversion to and from bytes should be on the
> encoded non-byte type, and other types including user created ones could
> follow that pattern. That may allow bytes to work with any type that has the
> required special methods to do the conversions.
>
> Then most of the methods on bytes would be for manipulating bytes in various
> ways.
>
>
> The constructor for the int type already does base and string conversions,
> extending it to bytes seems like it would be natural.
>
> ? int(bytes) ? ? ? ? ? ?# just like int(string)
> ? bytes = bytes(int) ? ?# calls int.__to_bytes__() to do the actual work.
>
>
> Ron
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From ede at mit.edu  Tue Aug 11 00:38:51 2009
From: ede at mit.edu (Eric Eisner)
Date: Tue, 11 Aug 2009 07:38:51 +0900
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <4A8055E2.2040001@ronadam.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com>
	<4A8055E2.2040001@ronadam.com>
Message-ID: <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com>

On Tue, Aug 11, 2009 at 02:16, Ron Adam<rrr at ronadam.com> wrote:
> The constructor for the int type already does base and string conversions,
> extending it to bytes seems like it would be natural.
>
> ? int(bytes) ? ? ? ? ? ?# just like int(string)
> ? bytes = bytes(int) ? ?# calls int.__to_bytes__() to do the actual work.

The constructor for bytes currently accepts a single int as an
argument, producing that many zero bytes. As far as I can tell this
behavior is undocumented, but I don't know if that can be changed
easily...

The first thing I did when I tried out the bytes type in 3.x was to
try to convert an int to a byte via the constructor, and the current
behavior surprised me.

-Eric


From greg.ewing at canterbury.ac.nz  Tue Aug 11 03:05:26 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 11 Aug 2009 13:05:26 +1200
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <4A801D0C.2000502@gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
Message-ID: <4A80C3D6.4030202@canterbury.ac.nz>

Nick Coghlan wrote:

> One slight quirk of the C API that probably shouldn't be replicated is a
> size of 0 translating to an integer result of zero. For the Python API,
> passing in an empty byte sequence should trigger a ValueError.

Why? This seems like a perfectly logical limiting case
to me.

> int.from_bytes(data, *, little_endian=None, signed=True)
>   little_endian would become a three-valued parameter for the Python
> version:  None = native; False = little-endian; True = big-endian.

I don't like the idea of a three-valued boolean. I also
don't like boolean parameters whose sense is abritrary
(why is it called "little_endian" and not "big_endian",
and how do I remember which convention was chosen?)

My suggestion would be to use the same characters that
the struct module uses to represent endianness (">"
for big-endian, "<" for little-endian, etc.)


-- 
Greg


From greg.ewing at canterbury.ac.nz  Tue Aug 11 03:08:57 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 11 Aug 2009 13:08:57 +1200
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <4A802671.9040002@gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com> <loom.20090810T133442-435@post.gmane.org>
	<4A802671.9040002@gmail.com>
Message-ID: <4A80C4A9.7010300@canterbury.ac.nz>

Nick Coghlan wrote:

> As far as bytearray goes, my original post did suggest that approach
> (i.e. class methods on the target types), but that then gets tedious as
> every new type that wants to support conversion from an integer needs to
> either go through one of the existing types or else have it's own C
> extension call the underlying C API directly.

I'd suggest that int.to_bytes should take an optional destination
parameter, and it should accept any object having a mutable
buffer interface.

That would keep all the int-related methods with the int class,
without restricting the source or destination to any particular
type.

-- 
Greg


From rrr at ronadam.com  Tue Aug 11 05:26:59 2009
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 10 Aug 2009 22:26:59 -0500
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>	
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>	
	<200908081554.36358.steve@pearwood.info>	
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>	
	<5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com>	
	<4A8055E2.2040001@ronadam.com>
	<42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com>
Message-ID: <4A80E503.7090407@ronadam.com>



Eric Eisner wrote:
> On Tue, Aug 11, 2009 at 02:16, Ron Adam<rrr at ronadam.com> wrote:
>> The constructor for the int type already does base and string conversions,
>> extending it to bytes seems like it would be natural.
>>
>>   int(bytes)            # just like int(string)
>>   bytes = bytes(int)    # calls int.__to_bytes__() to do the actual work.
> 
> The constructor for bytes currently accepts a single int as an
> argument, producing that many zero bytes. As far as I can tell this
> behavior is undocumented, but I don't know if that can be changed
> easily...
> 
> The first thing I did when I tried out the bytes type in 3.x was to
> try to convert an int to a byte via the constructor, and the current
> behavior surprised me.


It seems that bytes are quite different here in 2.6 and 3.0.

In 2.6 and bytes is an alias for string(), so the bytes constructor 
behavior just converts the int to a string.

ra at Gutsy:~$ python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> bytes(123)
'123'
 >>> bytes
<type 'str'>
 >>> help(bytes)

class str(basestring)
  |  str(object) -> string
  |
  |  Return a nice string representation of the object.
  |  If the argument is a string, the return value is the same object.
  |
(clipped)



Ron





From g.brandl at gmx.net  Tue Aug 11 08:01:55 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 11 Aug 2009 08:01:55 +0200
Subject: [Python-ideas] bitwise operations on bytes
In-Reply-To: <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>	<200908081554.36358.steve@pearwood.info>	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>	<5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com>	<4A8055E2.2040001@ronadam.com>
	<42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com>
Message-ID: <h5r1j4$8op$1@ger.gmane.org>

Eric Eisner schrieb:
> On Tue, Aug 11, 2009 at 02:16, Ron Adam<rrr at ronadam.com> wrote:
>> The constructor for the int type already does base and string conversions,
>> extending it to bytes seems like it would be natural.
>>
>>   int(bytes)            # just like int(string)
>>   bytes = bytes(int)    # calls int.__to_bytes__() to do the actual work.
> 
> The constructor for bytes currently accepts a single int as an
> argument, producing that many zero bytes. As far as I can tell this
> behavior is undocumented, but I don't know if that can be changed
> easily...

It is documented, see http://docs.python.org/py3k/library/functions#bytearray.

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From solipsis at pitrou.net  Tue Aug 11 12:43:23 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 11 Aug 2009 10:43:23 +0000 (UTC)
Subject: [Python-ideas] What about allowing '?' in method names?
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<4A7BEF80.1090505@canterbury.ac.nz>
	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>
	<200908081153.39137.steve@pearwood.info>
Message-ID: <loom.20090811T103808-371@post.gmane.org>

Steven D'Aprano <steve at ...> writes:
> 
> What makes you think that the question mark is a clue-in that a yes/no 
> answer is expected?

AFAIK it is a widely-used convention in the Ruby world.
I'd even go as far as saying that it's quite pretty, as a typographical
convention (not that other Ruby conventions are :-)).

> In any case, I don't think that "returns a boolean flag" versus "returns 
> a non-flag" is an important distinction.

I don't think this is a valid argument here. If '?' were to be allowed in
identifier names (or at least at their end), people would be free to choose
whatever convention they like.




From arnodel at googlemail.com  Tue Aug 11 15:25:15 2009
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Tue, 11 Aug 2009 14:25:15 +0100
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <loom.20090811T103808-371@post.gmane.org>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<4A7BEF80.1090505@canterbury.ac.nz>
	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>
	<200908081153.39137.steve@pearwood.info>
	<loom.20090811T103808-371@post.gmane.org>
Message-ID: <D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>


On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:

> Steven D'Aprano <steve at ...> writes:
>>
>> What makes you think that the question mark is a clue-in that a yes/ 
>> no
>> answer is expected?
>
> AFAIK it is a widely-used convention in the Ruby world.
> I'd even go as far as saying that it's quite pretty, as a  
> typographical
> convention (not that other Ruby conventions are :-))

Also in Scheme. (I think the question mark more or less replaces the  
'p' suffix used in LISP).
A noun, adjective or verb followed by a question mark is pretty  
obviously a predicate:

    number?
    empty?
    restart?

I expect them all to return a boolean.

-- 
Arnaud



From masklinn at masklinn.net  Tue Aug 11 15:55:04 2009
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 11 Aug 2009 15:55:04 +0200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<4A7BEF80.1090505@canterbury.ac.nz>
	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>
	<200908081153.39137.steve@pearwood.info>
	<loom.20090811T103808-371@post.gmane.org>
	<D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>
Message-ID: <C93BBA37-BE4D-4762-A2AB-9480E42C615B@masklinn.net>

On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote:
> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:
> Steven D'Aprano <steve at ...> writes:
>>>
>>> What makes you think that the question mark is a clue-in that a  
>>> yes/no
>>> answer is expected?
>>
>> AFAIK it is a widely-used convention in the Ruby world.
>> I'd even go as far as saying that it's quite pretty, as a  
>> typographical
>> convention (not that other Ruby conventions are :-))
>
> Also in Scheme. (I think the question mark more or less replaces the  
> 'p' suffix used in LISP).
It does, and Ruby's idea of using the "?" postfix for boolean query  
(instead of an `is` or `is_` prefix) comes from there.



From g.brandl at gmx.net  Tue Aug 11 19:33:13 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 11 Aug 2009 19:33:13 +0200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <C93BBA37-BE4D-4762-A2AB-9480E42C615B@masklinn.net>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>	<4A7BEF80.1090505@canterbury.ac.nz>	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>	<200908081153.39137.steve@pearwood.info>	<loom.20090811T103808-371@post.gmane.org>	<D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>
	<C93BBA37-BE4D-4762-A2AB-9480E42C615B@masklinn.net>
Message-ID: <h5sa3a$eek$1@ger.gmane.org>

Masklinn schrieb:
> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote:
>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:
>> Steven D'Aprano <steve at ...> writes:
>>>>
>>>> What makes you think that the question mark is a clue-in that a  
>>>> yes/no
>>>> answer is expected?
>>>
>>> AFAIK it is a widely-used convention in the Ruby world.
>>> I'd even go as far as saying that it's quite pretty, as a  
>>> typographical
>>> convention (not that other Ruby conventions are :-))
>>
>> Also in Scheme. (I think the question mark more or less replaces the  
>> 'p' suffix used in LISP).
> It does, and Ruby's idea of using the "?" postfix for boolean query  
> (instead of an `is` or `is_` prefix) comes from there.

But, and I believe that was Steven's point, it is no more than a convention.
Omitting the "is_" prefix makes for a "question" that isn't unambiguously
a yes/no question anymore in many cases.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From aaron.rubin at 4dtechnology.com  Tue Aug 11 20:08:53 2009
From: aaron.rubin at 4dtechnology.com (Aaron Rubin)
Date: Tue, 11 Aug 2009 11:08:53 -0700
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <200908081153.39137.steve@pearwood.info>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> 
	<4A7BEF80.1090505@canterbury.ac.nz>
	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> 
	<200908081153.39137.steve@pearwood.info>
Message-ID: <985db6080908111108l4dce84f4k5ef77afc2afde176@mail.gmail.com>

On Fri, Aug 7, 2009 at 6:53 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On Sat, 8 Aug 2009 11:07:30 am Tennessee Leeuwenburg wrote:
> > Hi all,
> > Well, the thread's basically concluded, but I'm not sure if it's
> > appropriate to just 'leave it hanging' seeing as I instigated the
> > thread. So thanks all for the discussion points and I think we can
> > wrap things up now. Please feel free to comment on the summary below
> > if you think it doesn't have all the relevant details.
> >
> > To summarise the discussion for posterity:
> >
> > On the Plus Side:
> >    ? is part of English grammar, and is an intuitive clue-in that a
> > yes/no, true/false response is expected as a result.
>
> What makes you think that the question mark is a clue-in that a yes/no
> answer is expected?


I think was the poster implied is that when a question mark is appended to
any statement (or word for that matter), it becomes an existence or yes/no
question.  There are exceptions, but they need the qualifiers "when, who,
where, why, etc." prepended (as you have proven below).  Otherwise, they are
Boolean.  Method or function names (the subject of this thread) are
typically nouns or verbs, making the point more poignant.


>
> When did you come up with the idea?
>
> Who would agree with you?
>
> Where can I see some evidence for that suggestion?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090811/a72f6880/attachment.html>

From masklinn at masklinn.net  Tue Aug 11 21:05:54 2009
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 11 Aug 2009 21:05:54 +0200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <h5sa3a$eek$1@ger.gmane.org>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>	<4A7BEF80.1090505@canterbury.ac.nz>	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>	<200908081153.39137.steve@pearwood.info>	<loom.20090811T103808-371@post.gmane.org>	<D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>
	<C93BBA37-BE4D-4762-A2AB-9480E42C615B@masklinn.net>
	<h5sa3a$eek$1@ger.gmane.org>
Message-ID: <CD50E6A5-D0B7-498A-AF8E-23FBCD311A64@masklinn.net>

On 11 Aug 2009, at 19:33 , Georg Brandl wrote:
> Masklinn schrieb:
>> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote:
>>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:
>>> Steven D'Aprano <steve at ...> writes:
>>>>>
>>>>> What makes you think that the question mark is a clue-in that a
>>>>> yes/no
>>>>> answer is expected?
>>>>
>>>> AFAIK it is a widely-used convention in the Ruby world.
>>>> I'd even go as far as saying that it's quite pretty, as a
>>>> typographical
>>>> convention (not that other Ruby conventions are :-))
>>>
>>> Also in Scheme. (I think the question mark more or less replaces the
>>> 'p' suffix used in LISP).
>> It does, and Ruby's idea of using the "?" postfix for boolean query
>> (instead of an `is` or `is_` prefix) comes from there.
>
> But, and I believe that was Steven's point, it is no more than a  
> convention.
It isn't, any more than Python's `_` prefix convention, or its `self`  
argument for that matter. I'd heard the Python community was pretty  
big on smart conventions but I might be wrong.


From g.brandl at gmx.net  Tue Aug 11 21:09:07 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 11 Aug 2009 21:09:07 +0200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <CD50E6A5-D0B7-498A-AF8E-23FBCD311A64@masklinn.net>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>	<4A7BEF80.1090505@canterbury.ac.nz>	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>	<200908081153.39137.steve@pearwood.info>	<loom.20090811T103808-371@post.gmane.org>	<D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>	<C93BBA37-BE4D-4762-A2AB-9480E42C615B@masklinn.net>	<h5sa3a$eek$1@ger.gmane.org>
	<CD50E6A5-D0B7-498A-AF8E-23FBCD311A64@masklinn.net>
Message-ID: <h5sfn5$7ej$1@ger.gmane.org>

Masklinn schrieb:
> On 11 Aug 2009, at 19:33 , Georg Brandl wrote:
>> Masklinn schrieb:
>>> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote:
>>>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:
>>>> Steven D'Aprano <steve at ...> writes:
>>>>>>
>>>>>> What makes you think that the question mark is a clue-in that a
>>>>>> yes/no
>>>>>> answer is expected?
>>>>>
>>>>> AFAIK it is a widely-used convention in the Ruby world.
>>>>> I'd even go as far as saying that it's quite pretty, as a
>>>>> typographical
>>>>> convention (not that other Ruby conventions are :-))
>>>>
>>>> Also in Scheme. (I think the question mark more or less replaces the
>>>> 'p' suffix used in LISP).
>>> It does, and Ruby's idea of using the "?" postfix for boolean query
>>> (instead of an `is` or `is_` prefix) comes from there.
>>
>> But, and I believe that was Steven's point, it is no more than a  
>> convention.
> It isn't, any more than Python's `_` prefix convention, or its `self`  
> argument for that matter. I'd heard the Python community was pretty  
> big on smart conventions but I might be wrong.

But what is the advantage of "?" to "is_" then?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.



From masklinn at masklinn.net  Tue Aug 11 21:56:24 2009
From: masklinn at masklinn.net (Masklinn)
Date: Tue, 11 Aug 2009 21:56:24 +0200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <h5sfn5$7ej$1@ger.gmane.org>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>	<4A7BEF80.1090505@canterbury.ac.nz>	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>	<200908081153.39137.steve@pearwood.info>	<loom.20090811T103808-371@post.gmane.org>	<D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>	<C93BBA37-BE4D-4762-A2AB-9480E42C615B@masklinn.net>	<h5sa3a$eek$1@ger.gmane.org>
	<CD50E6A5-D0B7-498A-AF8E-23FBCD311A64@masklinn.net>
	<h5sfn5$7ej$1@ger.gmane.org>
Message-ID: <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net>

On 11 Aug 2009, at 21:09 , Georg Brandl wrote:
> Masklinn schrieb:
>> On 11 Aug 2009, at 19:33 , Georg Brandl wrote:
>>> Masklinn schrieb:
>>>> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote:
>>>>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:
>>>>> Steven D'Aprano <steve at ...> writes:
>>>>>>>
>>>>>>> What makes you think that the question mark is a clue-in that a
>>>>>>> yes/no
>>>>>>> answer is expected?
>>>>>>
>>>>>> AFAIK it is a widely-used convention in the Ruby world.
>>>>>> I'd even go as far as saying that it's quite pretty, as a
>>>>>> typographical
>>>>>> convention (not that other Ruby conventions are :-))
>>>>>
>>>>> Also in Scheme. (I think the question mark more or less replaces  
>>>>> the
>>>>> 'p' suffix used in LISP).
>>>> It does, and Ruby's idea of using the "?" postfix for boolean query
>>>> (instead of an `is` or `is_` prefix) comes from there.
>>>
>>> But, and I believe that was Steven's point, it is no more than a
>>> convention.
>> It isn't, any more than Python's `_` prefix convention, or its `self`
>> argument for that matter. I'd heard the Python community was pretty
>> big on smart conventions but I might be wrong.
>
> But what is the advantage of "?" to "is_" then?
>
I think it's easier to spot due to 1. being a fairly rare character in  
programs (in languages where the ?: ternary doesn't exist anyway) and  
2. being at the end of the call (though on Python method the ()  
operator makes that much less interesting, if you aren't using a  
property).

is_ is less easy to spot as it's a pretty unremarkable sequence of  
characters and at the middle of the call/line.



From jnoller at gmail.com  Wed Aug 12 01:50:17 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 11 Aug 2009 19:50:17 -0400
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>
	<200908081153.39137.steve@pearwood.info>
	<loom.20090811T103808-371@post.gmane.org>
	<D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>
	<C93BBA37-BE4D-4762-A2AB-9480E42C615B@masklinn.net>
	<h5sa3a$eek$1@ger.gmane.org>
	<CD50E6A5-D0B7-498A-AF8E-23FBCD311A64@masklinn.net>
	<h5sfn5$7ej$1@ger.gmane.org>
	<27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net>
Message-ID: <4222a8490908111650i339f2a1bwba98cbb3f7652134@mail.gmail.com>

On Tue, Aug 11, 2009 at 3:56 PM, Masklinn<masklinn at masklinn.net> wrote:
> On 11 Aug 2009, at 21:09 , Georg Brandl wrote:
>>
>> Masklinn schrieb:
>>>
>>> On 11 Aug 2009, at 19:33 , Georg Brandl wrote:
>>>>
>>>> Masklinn schrieb:
>>>>>
>>>>> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote:
>>>>>>
>>>>>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:
>>>>>> Steven D'Aprano <steve at ...> writes:
>>>>>>>>
>>>>>>>> What makes you think that the question mark is a clue-in that a
>>>>>>>> yes/no
>>>>>>>> answer is expected?
>>>>>>>
>>>>>>> AFAIK it is a widely-used convention in the Ruby world.
>>>>>>> I'd even go as far as saying that it's quite pretty, as a
>>>>>>> typographical
>>>>>>> convention (not that other Ruby conventions are :-))
>>>>>>
>>>>>> Also in Scheme. (I think the question mark more or less replaces the
>>>>>> 'p' suffix used in LISP).
>>>>>
>>>>> It does, and Ruby's idea of using the "?" postfix for boolean query
>>>>> (instead of an `is` or `is_` prefix) comes from there.
>>>>
>>>> But, and I believe that was Steven's point, it is no more than a
>>>> convention.
>>>
>>> It isn't, any more than Python's `_` prefix convention, or its `self`
>>> argument for that matter. I'd heard the Python community was pretty
>>> big on smart conventions but I might be wrong.
>>
>> But what is the advantage of "?" to "is_" then?
>>
> I think it's easier to spot due to 1. being a fairly rare character in
> programs (in languages where the ?: ternary doesn't exist anyway) and 2.
> being at the end of the call (though on Python method the () operator makes
> that much less interesting, if you aren't using a property).
>
> is_ is less easy to spot as it's a pretty unremarkable sequence of
> characters and at the middle of the call/line.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>

Isn't this discussion moot? Guido already took it out back and sent it
on it's way.

jesse


From steve at pearwood.info  Wed Aug 12 02:21:19 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 12 Aug 2009 10:21:19 +1000
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<h5sfn5$7ej$1@ger.gmane.org>
	<27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net>
Message-ID: <200908121021.19273.steve@pearwood.info>

On Wed, 12 Aug 2009 05:56:24 am Masklinn wrote:

> >> I'd heard the Python community
> >> was pretty big on smart conventions but I might be wrong.

And would accepting or rejecting this proposal prove that we're big on 
smart conventions?


> > But what is the advantage of "?" to "is_" then?
>
> I think it's easier to spot due to 1. being a fairly rare character
> in programs (in languages where the ?: ternary doesn't exist anyway)

It's certainly rare in Python, because it's illegal (except in strings 
and comments). It won't be rare if this convention is allowed.


> and 2. being at the end of the call (though on Python method the ()
> operator makes that much less interesting, if you aren't using a
> property).

A thought experiment -- what if this convention had been in place in 
Python? Here is literally the first piece of code I looked at from 
doctest.py with a Yes/No test in it:

def _normalize_module(module, depth=2):
    if inspect.module?(module):
        return module
    elif instance?(module, (str, unicode)):
        return __import__(module, globals(), locals(), ["*"])
    elif module is None:
        return sys.modules[sys._getframe(depth).f_globals['__name__']]
    else:
        raise TypeError("Expected a module, string, or None")

Notice the inconsistency -- method and function calls which return a 
bool have a question mark, but operators do not. I hope you're not 
going to suggest changing `is` to `is?`.

But in any case, I don't think instance?() is any more readable than 
isinstance(), or module?() than ismodule(). Perhaps one advantage is 
that you can run your spell checker over your code with fewer false 
negatives.


Deeper in doctest.py:

    def _failure_header(self, test, example):
        out = [self.DIVIDER]
        if test.filename:
            if test.lineno is not None and example.lineno is not None:
                lineno = test.lineno + example.lineno + 1
            ...

Allowing ? in identifiers won't change this method, but it illustrates 
another characteristic of Python (and Pythonic) code: the distinction 
between Yes/No booleans and Everything Else simply isn't that 
important. There's a method with two tests, one (the inner) is a 
conventional True or False test, but the other (the outer) is a 
Something or Nothing style test. 

For example, str.startswith(prefix) and str.endswith(suffix) could be 
modified to return the prefix/suffix found, or the empty string, and 
virtually all code using them should continue to work correctly. The 
exceptions being code that did something like this:

if "a string".startswith("a") is True:
    ...

but in my opinion, code that bad deserves to be broken.

Elevating functions that return a Yes/No answer over other functions 
with extra punctuation is an unnecessary, and dare I say, un-pythonic, 
distinction.


> is_ is less easy to spot as it's a pretty unremarkable sequence of
> characters and at the middle of the call/line.

The same argument could be made for len, str, dict, or any other word. 
You can't expect to understand code without reading the words. Adding 
another punctuation character identifiers isn't going to change that.



-- 
Steven D'Aprano


From masklinn at masklinn.net  Wed Aug 12 09:32:01 2009
From: masklinn at masklinn.net (Masklinn)
Date: Wed, 12 Aug 2009 09:32:01 +0200
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <200908121021.19273.steve@pearwood.info>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<h5sfn5$7ej$1@ger.gmane.org>
	<27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net>
	<200908121021.19273.steve@pearwood.info>
Message-ID: <7ED7005D-BDE5-4504-BE49-891A27686225@masklinn.net>

On 12 Aug 2009, at 02:21 , Steven D'Aprano wrote:
> On Wed, 12 Aug 2009 05:56:24 am Masklinn wrote:
>
>>>> I'd heard the Python community
>>>> was pretty big on smart conventions but I might be wrong.
>
> And would accepting or rejecting this proposal prove that we're big on
> smart conventions?
>
That's not what I wrote.

>>> But what is the advantage of "?" to "is_" then?
>>
>> I think it's easier to spot due to 1. being a fairly rare character
>> in programs (in languages where the ?: ternary doesn't exist anyway)
>
> It's certainly rare in Python, because it's illegal (except in strings
> and comments).
Which is exactly what I said, but thanks for repeating it.

> Notice the inconsistency -- method and function calls which return a
> bool have a question mark, but operators do not. I hope you're not
> going to suggest changing `is` to `is?`.
>
That's an interesting point.


From larry at hastings.org  Wed Aug 12 16:32:01 2009
From: larry at hastings.org (Larry Hastings)
Date: Wed, 12 Aug 2009 07:32:01 -0700
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>	<4A7BEF80.1090505@canterbury.ac.nz>	<43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com>	<200908081153.39137.steve@pearwood.info>	<loom.20090811T103808-371@post.gmane.org>
	<D6ED840B-4089-4EF7-8084-B32F2544DF44@googlemail.com>
Message-ID: <4A82D261.5020404@hastings.org>

Arnaud Delobelle wrote:
> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote:
>> AFAIK it is a widely-used convention in the Ruby world.
> Also in Scheme.

And also in FORTH.  See page 174 of the "Thinking FORTH" PDF, online 
version of the 1984 book.  Or on this classic bumper sticker:

FORTH LOVE? IF HONK THEN


/larry/


From ceronman at gmail.com  Wed Aug 12 20:02:22 2009
From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=)
Date: Wed, 12 Aug 2009 13:02:22 -0500
Subject: [Python-ideas] Adding some standalone iterator/sequence functions
	as methods of the iterator objects
Message-ID: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>

Hello.

There are some functions in the standard library that take an
iterator/sequence as parameter and return an iterator. Most of them
are in the itertools module and some are built in functions. I think
they should be added as methods of the iterator objects as well. For
example:

itertools.takewhile(pred, seq) --> seq.takewhile(pred)

sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun, reverse)


Rationale:
=======

First, I know the rationale behind standalone functions like len as
opposed to methods, but I think some iterator functions are special
cases. I believe it is a common pattern to arrange these kind of
functions in a pipe-filter system to perform complex queries over
collections. The current system of standalone functions creates code
difficult to read with nested parenthesis:

...fun4(param, fun3(param, fun2(param, fun1(param, seq))))...

It is very hard to see the pipe-filter flow in this code. The case is
even worse because in some functions the order of the sequence
argument and other parameters vary. For example: sorted takes the
sequence first and then the key and reverse parameters while
itertools.takewhile takes the predicate first and then the sequence.

A few months ago, Donald 'Paddy' McCarthy suggested a pipe function
[0] in the itertools module. But I believe using methods creates a
better work flow, for example:

seq.fun1(param).fun2(param).fun3(param).fun4(param)

[0] http://mail.python.org/pipermail/python-ideas/2009-May/004877.html


Examples:
========

Example 1. I want two groups of employees with the two best salaries:

Using current functions:

groups = itertools.islice(itertools.groupby(sorted(employees,
key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2)

Using methods:

groups = employees.sorted(lambda e: e.salary,
reverse=True).groupby(lambda e: e.salary).slice(None, 2)

Example 2. I want the pairs of programmers assigned by task:

Using current functions:

pairs_tasks = itertools.izip(itertools.cycle(itertools.combinations(programmers,
2)), tasks)

Using methods:

pars_tasks = programmers.combinations(2).cycle().izip(tasks)

Probably is better to keep izip as a standalone function:

pars_tasks = itertools.izip(programmers.combinations(2).cycle(), tasks)


Precedent:
========

There is another case where the pipe-filter pattern is seen in Python:
strings. There are a lot of functions in the string module that take
strings as argument and returns a string. Those functions could be
arranged in a pipe-filter system. Python has a history of adding
functions from the string module to the string objects. I think the
same could be done with iterator functions.

Example:

We can use:

parts = text.lower().strip().split()

As opposed to:

parts = string.split(string.strip(string.lower(text)))


That's all for now. If you think this is a good idea we could
elaborate on which methods should be added.

Hope to see your comments.

Manuel Cer?n.


From guido at python.org  Wed Aug 12 20:05:26 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 12 Aug 2009 11:05:26 -0700
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
Message-ID: <ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>

No, no, no!

This is intentional. The iterator protocol is intentionally minimal --
there are many implementations of it, and those implementations don't
share code. Adding a large number of methods to it would require every
iterator implementation to provide all those methods.

This has been discussed before. Maybe it's time to write a negative
PEP? Or add a paragraph explaining this to the original iterator
protocol PEP? Or to the itertools module docs?

--Guido

2009/8/12 Manuel Alejandro Cer?n Estrada <ceronman at gmail.com>:
> Hello.
>
> There are some functions in the standard library that take an
> iterator/sequence as parameter and return an iterator. Most of them
> are in the itertools module and some are built in functions. I think
> they should be added as methods of the iterator objects as well. For
> example:
>
> itertools.takewhile(pred, seq) --> seq.takewhile(pred)
>
> sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun, reverse)
>
>
> Rationale:
> =======
>
> First, I know the rationale behind standalone functions like len as
> opposed to methods, but I think some iterator functions are special
> cases. I believe it is a common pattern to arrange these kind of
> functions in a pipe-filter system to perform complex queries over
> collections. The current system of standalone functions creates code
> difficult to read with nested parenthesis:
>
> ...fun4(param, fun3(param, fun2(param, fun1(param, seq))))...
>
> It is very hard to see the pipe-filter flow in this code. The case is
> even worse because in some functions the order of the sequence
> argument and other parameters vary. For example: sorted takes the
> sequence first and then the key and reverse parameters while
> itertools.takewhile takes the predicate first and then the sequence.
>
> A few months ago, Donald 'Paddy' McCarthy suggested a pipe function
> [0] in the itertools module. But I believe using methods creates a
> better work flow, for example:
>
> seq.fun1(param).fun2(param).fun3(param).fun4(param)
>
> [0] http://mail.python.org/pipermail/python-ideas/2009-May/004877.html
>
>
> Examples:
> ========
>
> Example 1. I want two groups of employees with the two best salaries:
>
> Using current functions:
>
> groups = itertools.islice(itertools.groupby(sorted(employees,
> key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2)
>
> Using methods:
>
> groups = employees.sorted(lambda e: e.salary,
> reverse=True).groupby(lambda e: e.salary).slice(None, 2)
>
> Example 2. I want the pairs of programmers assigned by task:
>
> Using current functions:
>
> pairs_tasks = itertools.izip(itertools.cycle(itertools.combinations(programmers,
> 2)), tasks)
>
> Using methods:
>
> pars_tasks = programmers.combinations(2).cycle().izip(tasks)
>
> Probably is better to keep izip as a standalone function:
>
> pars_tasks = itertools.izip(programmers.combinations(2).cycle(), tasks)
>
>
> Precedent:
> ========
>
> There is another case where the pipe-filter pattern is seen in Python:
> strings. There are a lot of functions in the string module that take
> strings as argument and returns a string. Those functions could be
> arranged in a pipe-filter system. Python has a history of adding
> functions from the string module to the string objects. I think the
> same could be done with iterator functions.
>
> Example:
>
> We can use:
>
> parts = text.lower().strip().split()
>
> As opposed to:
>
> parts = string.split(string.strip(string.lower(text)))
>
>
> That's all for now. If you think this is a good idea we could
> elaborate on which methods should be added.
>
> Hope to see your comments.
>
> Manuel Cer?n.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From jkwon.work at gmail.com  Wed Aug 12 22:11:47 2009
From: jkwon.work at gmail.com (Jae Kwon)
Date: Wed, 12 Aug 2009 13:11:47 -0700
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>
Message-ID: <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>

these methods should be available in a common superclass (not the  
iterator protocol), and
all or most python native iterators could subclass it.

On Aug 12, 2009, at 11:05 AM, Guido van Rossum wrote:

> No, no, no!
>
> This is intentional. The iterator protocol is intentionally minimal --
> there are many implementations of it, and those implementations don't
> share code. Adding a large number of methods to it would require every
> iterator implementation to provide all those methods.
>
> This has been discussed before. Maybe it's time to write a negative
> PEP? Or add a paragraph explaining this to the original iterator
> protocol PEP? Or to the itertools module docs?
>
> --Guido
>
> 2009/8/12 Manuel Alejandro Cer?n Estrada <ceronman at gmail.com>:
>> Hello.
>>
>> There are some functions in the standard library that take an
>> iterator/sequence as parameter and return an iterator. Most of them
>> are in the itertools module and some are built in functions. I think
>> they should be added as methods of the iterator objects as well. For
>> example:
>>
>> itertools.takewhile(pred, seq) --> seq.takewhile(pred)
>>
>> sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun,  
>> reverse)
>>
>>
>> Rationale:
>> =======
>>
>> First, I know the rationale behind standalone functions like len as
>> opposed to methods, but I think some iterator functions are special
>> cases. I believe it is a common pattern to arrange these kind of
>> functions in a pipe-filter system to perform complex queries over
>> collections. The current system of standalone functions creates code
>> difficult to read with nested parenthesis:
>>
>> ...fun4(param, fun3(param, fun2(param, fun1(param, seq))))...
>>
>> It is very hard to see the pipe-filter flow in this code. The case is
>> even worse because in some functions the order of the sequence
>> argument and other parameters vary. For example: sorted takes the
>> sequence first and then the key and reverse parameters while
>> itertools.takewhile takes the predicate first and then the sequence.
>>
>> A few months ago, Donald 'Paddy' McCarthy suggested a pipe function
>> [0] in the itertools module. But I believe using methods creates a
>> better work flow, for example:
>>
>> seq.fun1(param).fun2(param).fun3(param).fun4(param)
>>
>> [0] http://mail.python.org/pipermail/python-ideas/2009-May/ 
>> 004877.html
>>
>>
>> Examples:
>> ========
>>
>> Example 1. I want two groups of employees with the two best salaries:
>>
>> Using current functions:
>>
>> groups = itertools.islice(itertools.groupby(sorted(employees,
>> key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2)
>>
>> Using methods:
>>
>> groups = employees.sorted(lambda e: e.salary,
>> reverse=True).groupby(lambda e: e.salary).slice(None, 2)
>>
>> Example 2. I want the pairs of programmers assigned by task:
>>
>> Using current functions:
>>
>> pairs_tasks =  
>> itertools.izip(itertools.cycle(itertools.combinations(programmers,
>> 2)), tasks)
>>
>> Using methods:
>>
>> pars_tasks = programmers.combinations(2).cycle().izip(tasks)
>>
>> Probably is better to keep izip as a standalone function:
>>
>> pars_tasks = itertools.izip(programmers.combinations(2).cycle(),  
>> tasks)
>>
>>
>> Precedent:
>> ========
>>
>> There is another case where the pipe-filter pattern is seen in  
>> Python:
>> strings. There are a lot of functions in the string module that take
>> strings as argument and returns a string. Those functions could be
>> arranged in a pipe-filter system. Python has a history of adding
>> functions from the string module to the string objects. I think the
>> same could be done with iterator functions.
>>
>> Example:
>>
>> We can use:
>>
>> parts = text.lower().strip().split()
>>
>> As opposed to:
>>
>> parts = string.split(string.strip(string.lower(text)))
>>
>>
>> That's all for now. If you think this is a good idea we could
>> elaborate on which methods should be added.
>>
>> Hope to see your comments.
>>
>> Manuel Cer?n.
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>
>
>
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas



From guido at python.org  Wed Aug 12 22:12:43 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 12 Aug 2009 13:12:43 -0700
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> 
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com> 
	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>
Message-ID: <ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>

That idea has also been considered and rejected, because of Python's
commitment to duck typing.

Really, please find the old discussion and read it rather than
rehashing old broken ideas.

2009/8/12 Jae Kwon <jkwon.work at gmail.com>:
> these methods should be available in a common superclass (not the iterator
> protocol), and
> all or most python native iterators could subclass it.
>
> On Aug 12, 2009, at 11:05 AM, Guido van Rossum wrote:
>
>> No, no, no!
>>
>> This is intentional. The iterator protocol is intentionally minimal --
>> there are many implementations of it, and those implementations don't
>> share code. Adding a large number of methods to it would require every
>> iterator implementation to provide all those methods.
>>
>> This has been discussed before. Maybe it's time to write a negative
>> PEP? Or add a paragraph explaining this to the original iterator
>> protocol PEP? Or to the itertools module docs?
>>
>> --Guido
>>
>> 2009/8/12 Manuel Alejandro Cer?n Estrada <ceronman at gmail.com>:
>>>
>>> Hello.
>>>
>>> There are some functions in the standard library that take an
>>> iterator/sequence as parameter and return an iterator. Most of them
>>> are in the itertools module and some are built in functions. I think
>>> they should be added as methods of the iterator objects as well. For
>>> example:
>>>
>>> itertools.takewhile(pred, seq) --> seq.takewhile(pred)
>>>
>>> sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun, reverse)
>>>
>>>
>>> Rationale:
>>> =======
>>>
>>> First, I know the rationale behind standalone functions like len as
>>> opposed to methods, but I think some iterator functions are special
>>> cases. I believe it is a common pattern to arrange these kind of
>>> functions in a pipe-filter system to perform complex queries over
>>> collections. The current system of standalone functions creates code
>>> difficult to read with nested parenthesis:
>>>
>>> ...fun4(param, fun3(param, fun2(param, fun1(param, seq))))...
>>>
>>> It is very hard to see the pipe-filter flow in this code. The case is
>>> even worse because in some functions the order of the sequence
>>> argument and other parameters vary. For example: sorted takes the
>>> sequence first and then the key and reverse parameters while
>>> itertools.takewhile takes the predicate first and then the sequence.
>>>
>>> A few months ago, Donald 'Paddy' McCarthy suggested a pipe function
>>> [0] in the itertools module. But I believe using methods creates a
>>> better work flow, for example:
>>>
>>> seq.fun1(param).fun2(param).fun3(param).fun4(param)
>>>
>>> [0] http://mail.python.org/pipermail/python-ideas/2009-May/004877.html
>>>
>>>
>>> Examples:
>>> ========
>>>
>>> Example 1. I want two groups of employees with the two best salaries:
>>>
>>> Using current functions:
>>>
>>> groups = itertools.islice(itertools.groupby(sorted(employees,
>>> key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2)
>>>
>>> Using methods:
>>>
>>> groups = employees.sorted(lambda e: e.salary,
>>> reverse=True).groupby(lambda e: e.salary).slice(None, 2)
>>>
>>> Example 2. I want the pairs of programmers assigned by task:
>>>
>>> Using current functions:
>>>
>>> pairs_tasks =
>>> itertools.izip(itertools.cycle(itertools.combinations(programmers,
>>> 2)), tasks)
>>>
>>> Using methods:
>>>
>>> pars_tasks = programmers.combinations(2).cycle().izip(tasks)
>>>
>>> Probably is better to keep izip as a standalone function:
>>>
>>> pars_tasks = itertools.izip(programmers.combinations(2).cycle(), tasks)
>>>
>>>
>>> Precedent:
>>> ========
>>>
>>> There is another case where the pipe-filter pattern is seen in Python:
>>> strings. There are a lot of functions in the string module that take
>>> strings as argument and returns a string. Those functions could be
>>> arranged in a pipe-filter system. Python has a history of adding
>>> functions from the string module to the string objects. I think the
>>> same could be done with iterator functions.
>>>
>>> Example:
>>>
>>> We can use:
>>>
>>> parts = text.lower().strip().split()
>>>
>>> As opposed to:
>>>
>>> parts = string.split(string.strip(string.lower(text)))
>>>
>>>
>>> That's all for now. If you think this is a good idea we could
>>> elaborate on which methods should be added.
>>>
>>> Hope to see your comments.
>>>
>>> Manuel Cer?n.
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> http://mail.python.org/mailman/listinfo/python-ideas
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)


From ceronman at gmail.com  Wed Aug 12 22:48:29 2009
From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=)
Date: Wed, 12 Aug 2009 15:48:29 -0500
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> 
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com> 
	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>
	<ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>
Message-ID: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>

Hello again.

El 12 de agosto de 2009 15:12, Guido van Rossum<guido at python.org> escribi?:
> That idea has also been considered and rejected, because of Python's
> commitment to duck typing.
>
> Really, please find the old discussion and read it rather than
> rehashing old broken ideas.

Sorry for getting back already discussed ideas. Finding old
discussions can be very hard since the Python mailing lists have
thousands of mails and some time you just don't use the correct
keywords while searching.

I understand why this has been rejected. However, I still think that
the core idea of making the pipe-filter pattern and queries more
readable is valuable.

I'm thinking on implementing this as a third party wrapper module for
itertools and others. An QueryIter wrapper class could help to this
purpose and It would be very easy to implement. Could be something
like this:

pairs_tasks = IterQuery(programmers).combinations(2).cycle().izip(tasks)

Methods would be simple wrappers:

def combinations(self, repeat):
    return IterQuery(itertools.combinations(self, repeat))

Manuel.

-- 
Manuel Alejandro Cer?n Estrada
http://ceronman.freaks-unidos.net


From jkwon.work at gmail.com  Wed Aug 12 23:04:32 2009
From: jkwon.work at gmail.com (Jae Kwon)
Date: Wed, 12 Aug 2009 14:04:32 -0700
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>
	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>
	<ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>
	<796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
Message-ID: <6BB6141B-3E1C-44EF-A3E7-C2825759B85A@gmail.com>

+1

i tried to google "site:http://mail.python.org/pipermail/python-ideas/  
itertools" and could not find anything.

also, i like this idea.

  - Jae

On Aug 12, 2009, at 1:48 PM, Manuel Alejandro Cer?n Estrada wrote:

> Hello again.
>
> El 12 de agosto de 2009 15:12, Guido van Rossum<guido at python.org>  
> escribi?:
>> That idea has also been considered and rejected, because of Python's
>> commitment to duck typing.
>>
>> Really, please find the old discussion and read it rather than
>> rehashing old broken ideas.
>
> Sorry for getting back already discussed ideas. Finding old
> discussions can be very hard since the Python mailing lists have
> thousands of mails and some time you just don't use the correct
> keywords while searching.
>
> I understand why this has been rejected. However, I still think that
> the core idea of making the pipe-filter pattern and queries more
> readable is valuable.
>
> I'm thinking on implementing this as a third party wrapper module for
> itertools and others. An QueryIter wrapper class could help to this
> purpose and It would be very easy to implement. Could be something
> like this:
>
> pairs_tasks =  
> IterQuery(programmers).combinations(2).cycle().izip(tasks)
>
> Methods would be simple wrappers:
>
> def combinations(self, repeat):
>    return IterQuery(itertools.combinations(self, repeat))
>
> Manuel.
>
> -- 
> Manuel Alejandro Cer?n Estrada
> http://ceronman.freaks-unidos.net



From zuo at chopin.edu.pl  Thu Aug 13 01:32:19 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 13 Aug 2009 01:32:19 +0200
Subject: [Python-ideas] Adding some standalone iterator/sequence
 functions as methods of the iterator objects
In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>
	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>
	<ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>
	<796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
Message-ID: <op.uykoz4qtfvx12t@jerzozwiesz.home.aster.pl>

12-08-2009 Manuel Alejandro Cer?n Estrada <ceronman at gmail.com> wrote:

> I understand why this has been rejected. However, I still think that
> the core idea of making the pipe-filter pattern and queries more
> readable is valuable.
>
> I'm thinking on implementing this as a third party wrapper module for
> itertools and others. An QueryIter wrapper class could help to this
> purpose and It would be very easy to implement. Could be something
> like this:
>
> pairs_tasks = IterQuery(programmers).combinations(2).cycle().izip(tasks)
>
> Methods would be simple wrappers:
>
> def combinations(self, repeat):
>     return IterQuery(itertools.combinations(self, repeat))

Maybe it could be generalized? Imagine funpipe() factory function (or
with another name, doesn't matter at the moment) which could produce
proxy objects with such a feature that e.g.:

funpipe(<args0...>).fun1(<args1...>).fun2(<args2...>).fun3(<args3...>)

-- would be effectively equivalent to:

fun3(fun2(fun1(<args0...>, <args1...>), <args2...>), <args3...>)

-- where <args0...> means 0 or more positional arguments,
    and <args1-X...> means 0 or more positional and/or kw. arguments.

I hope I'm clear about what's the idea.

It could be in functools, itertools or as a built-in function...

What do you think?

*j
-- 
Jan Kaliszewski (zuo) <zuo at chopin.edu.pl>


From zuo at chopin.edu.pl  Thu Aug 13 01:40:14 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 13 Aug 2009 01:40:14 +0200
Subject: [Python-ideas] Adding some standalone iterator/sequence
 functions as methods of the iterator objects
In-Reply-To: <op.uykoz4qtfvx12t@jerzozwiesz.home.aster.pl>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>
	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>
	<ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>
	<796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
	<op.uykoz4qtfvx12t@jerzozwiesz.home.aster.pl>
Message-ID: <op.uykpdcahfvx12t@jerzozwiesz.home.aster.pl>

13-08-2009 Jan Kaliszewski <zuo at chopin.edu.pl> wrote:

> Maybe it could be generalized? Imagine funpipe() factory function (or
> with another name, doesn't matter at the moment) which could produce
> proxy objects with such a feature that e.g.:
>
> funpipe(<args0...>).fun1(<args1...>).fun2(<args2...>).fun3(<args3...>)


PS. Maybe '|' operator would be here more easy to implement and more
logical -- e.g.:

funpipe().fun1(<args1...>) | fun2(<args2...>) | fun3(<args3...>)


Treat it as a 'loud thinking'.

*j
-- 
Jan Kaliszewski (zuo) <zuo at chopin.edu.pl>


From george.sakkis at gmail.com  Thu Aug 13 02:18:27 2009
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 12 Aug 2009 20:18:27 -0400
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>
	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>
	<ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>
	<796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
Message-ID: <91ad5bf80908121718u473ece31o93e56ea080557a6c@mail.gmail.com>

2009/8/12 Manuel Alejandro Cer?n Estrada <ceronman at gmail.com>:

> Sorry for getting back already discussed ideas. Finding old
> discussions can be very hard since the Python mailing lists have
> thousands of mails and some time you just don't use the correct
> keywords while searching.

Check out the (long) "Builtin iterator type" thread at
http://mail.python.org/pipermail/python-3000/2006-November/; I don't
know if there are more.

> I'm thinking on implementing this as a third party wrapper module for
> itertools and others. An QueryIter wrapper class could help to this
> purpose and It would be very easy to implement. Could be something
> like this:
>
> pairs_tasks = IterQuery(programmers).combinations(2).cycle().izip(tasks)

You can start from the recipe spawned off the 2006 thread:
http://code.activestate.com/recipes/498272/.

George


From ede at mit.edu  Thu Aug 13 03:25:24 2009
From: ede at mit.edu (Eric Eisner)
Date: Thu, 13 Aug 2009 10:25:24 +0900
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
Message-ID: <42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com>

2009/8/13 Manuel Alejandro Cer?n Estrada <ceronman at gmail.com>:
> Using current functions:
>
> pairs_tasks = itertools.izip(itertools.cycle(itertools.combinations(programmers,
> 2)), tasks)
>
> Using methods:
>
> pars_tasks = programmers.combinations(2).cycle().izip(tasks)

As an aside, I've always thought that Python tends to use functions
because Guido's native language is a head-initial language, where the
action precedes the object. On the other hand, Ruby tends to use
methods because its creator, Yukihiro Matsumoto, speaks a head-final
language, where the object preceds the action.

So in conclusion, functions are more pythonic.

-Eric


From tjreedy at udel.edu  Thu Aug 13 07:08:11 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 13 Aug 2009 14:08:11 +0900
Subject: [Python-ideas] Adding some standalone iterator/sequence
 functions as methods of the iterator objects
In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>
	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>	<ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>
	<796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
Message-ID: <h60740$ck2$1@ger.gmane.org>

Manuel Alejandro Cer?n Estrada wrote:

> I'm thinking on implementing this as a third party wrapper module for
> itertools and others.

For something that is longer than a recipe, that is definitely the way 
to go.  You are free to do it the way you want to and free to try 
variations. People who agree with you can give feedback based on real 
experience.

tjr



From ncoghlan at gmail.com  Thu Aug 13 11:31:59 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 13 Aug 2009 19:31:59 +1000
Subject: [Python-ideas] Adding some standalone
 iterator/sequence	functions as methods of the iterator objects
In-Reply-To: <6BB6141B-3E1C-44EF-A3E7-C2825759B85A@gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>	<ca471dc20908121105j76cb1b54y96ea6d4b58e520a6@mail.gmail.com>	<3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com>	<ca471dc20908121312n1443e58cm803aa53ddf848d26@mail.gmail.com>	<796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com>
	<6BB6141B-3E1C-44EF-A3E7-C2825759B85A@gmail.com>
Message-ID: <4A83DD8F.5000808@gmail.com>

Jae Kwon wrote:
> +1
> 
> i tried to google "site:http://mail.python.org/pipermail/python-ideas/
> itertools" and could not find anything.

python-ideas is a relatively young list - older discussions are all on
python-dev. It's usually best to search both lists anyway, since less
"out there" suggestions are often brought up on python-dev rather than here.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From janssen at parc.com  Thu Aug 13 18:23:08 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 13 Aug 2009 09:23:08 PDT
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com>
Message-ID: <10531.1250180588@parc.com>

Interesting thread.

Is Python procedural (function-oriented)?  My opinion is that it was,
when it started out, but isn't really any more, due to the growth of an
object-oriented style, in which data, state, is hidden behind an
RPC-like interface, finally officially "blessed" in Python 3 with the
appearance of abstract base classes.  In an alternate history, a more
procedural idea of Python could have grown generic functions, a la
Common Lisp.

Should the iterator interface be "wide" (lots of utility methods),
either by extending it directly or by producing a wide superclass?
Probably not.  Most languages with wide interfaces need them because
they only have "types", or they have objects, but only single
inheritance.  Java is only half-way there, with its notion of interfaces
(most of which are very wide (too wide), as well).  But in Python, you
can produce mixins, which are fully implemented partial classes,
intended to be mixed together with other classes for actual use.  So you
can produce lots of "narrow" interfaces, and mix them in as needed.
Keeping individual interfaces narrow also makes it easier to do
duck-typing.

Unfortunately, it seems to me that Python currently makes mixins
somewhat clumsy to use.  We could use a low-weight way of making a class
which is a mix of three or four mixins, much as "lambda" provides a
low-weight way of producing a small function.  Something like

    a = class(Mixin1, Mixin2, Mixin3)()
    a.mixin1_method_foo()
    a.mixin3_method_bar()

and so on.

Bill


From zuo at chopin.edu.pl  Thu Aug 13 20:32:36 2009
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Thu, 13 Aug 2009 20:32:36 +0200
Subject: [Python-ideas] Adding some standalone iterator/sequence
 functions as methods of the iterator objects
In-Reply-To: <10531.1250180588@parc.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com>
	<10531.1250180588@parc.com>
Message-ID: <op.uyl5sm1jfvx12t@jerzozwiesz.home.aster.pl>

13-08-2009 Bill Janssen <janssen at parc.com> wrote:

>     a = class(Mixin1, Mixin2, Mixin3)()
>     a.mixin1_method_foo()
>     a.mixin3_method_bar()

Something similar already is possible out of the box:

a = type('a', (Mixin1, Mixin2, Mixin3), {})()
Mixin1.foo(a)
Mixin3.bar(a)

You could also make creating of such objects more compact
by using prepared factory function, e.g.

def makeclass(*base_classes, **kwargs):
     cls_name = kwargs.get('name', 'noname')
     cls_attrs = kwargs.get('attrs', {})
     return type(cls_name, base_classes, cls_attrs)

Then you have exactly what you wanted:

a = makeclass(Mixin1, Mixin2, Mixin3)()


Regards,
*j
-- 
Jan Kaliszewski (zuo) <zuo at chopin.edu.pl>


From dreamingforward at gmail.com  Fri Aug 14 03:04:16 2009
From: dreamingforward at gmail.com (average)
Date: Thu, 13 Aug 2009 20:04:16 -0500
Subject: [Python-ideas] What about allowing '?' in method names?
In-Reply-To: <200908121021.19273.steve@pearwood.info>
References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com>
	<h5sfn5$7ej$1@ger.gmane.org>
	<27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net>
	<200908121021.19273.steve@pearwood.info>
Message-ID: <913f9f570908131804x3f11e443ie4fa8b10b511eee9@mail.gmail.com>

> And would accepting or rejecting this proposal prove that we're big on
> smart conventions?
>
>> > But what is the advantage of "?" to "is_" then?
>>
>> I think it's easier to spot due to 1. being a fairly rare character
>> in programs (in languages where the ?: ternary doesn't exist anyway)

I know for myself, it would help me mentally organize all the
different methods within classes.  To me its the sort of syntactical
convention similar to using [] brackets to set off a conceptual item
as opposed to using parens and making a standard, more generic
function call.  Or like the elation of when the user finally gets the
point of whitespace and case conventions in python -- they've just
improved conceptual separation (and thus readability, maintainability)
by several-fold by simply agreeing to a widely-applicable convention.
The argument "Really, blah blah how difficult is it to just use is_ ?"
is insufficient.

To me it goes along with what's been learned with independant
component analysis (ICA)--elongate the distance between
information-independant items while minimizing the effort to keep
related information grouped together.  (ICA is the same tech that
allows one to separate conversations at a cocktail party.)  This
pretty much encapsulates the evolution of programming languages.

\> A thought experiment -- what if this convention had been in place in
> Python? Here is literally the first piece of code I looked at from
> doctest.py with a Yes/No test in it:
>
> def _normalize_module(module, depth=2):
> ? ?if inspect.module?(module):
> ? ? ? ?return module
> ? ?elif instance?(module, (str, unicode)):
> ? ? ? ?return __import__(module, globals(), locals(), ["*"])
> ? ?elif module is None:
> ? ? ? ?return sys.modules[sys._getframe(depth).f_globals['__name__']]
> ? ?else:
> ? ? ? ?raise TypeError("Expected a module, string, or None")

Note that any example on existing code will only be partially useful
as the names chosen were chosen before an option that provided greater
conceptual separation was available.

> Notice the inconsistency -- method and function calls which return a
> bool have a question mark, but operators do not. I hope you're not
> going to suggest changing `is` to `is?`.

That would be redundant.

In any case, I would take this suggestion for an additional convention
even further by suggesting that methods which have side effects or
mutate the object (list.sorted) should have an exclaimation point,
getting rid of ambiguous usages like list.sort and list.sorted which
are counter-intuitive--keep list.sort! for an in-place, mutating
method name, for example, and list.sort to return a sorted list.

Marcos


From prozacgod at gmail.com  Fri Aug 14 08:22:07 2009
From: prozacgod at gmail.com (Prozacgod)
Date: Fri, 14 Aug 2009 01:22:07 -0500
Subject: [Python-ideas] Interrogate alternate namespace keyword and concept
Message-ID: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>

A while back i had learned about the 'with' statement and at first I naively
thought that it worked similarly to a with statement I was familiar with,
from delphi/pascal - obviously it doesn't and I was instantly hit with the
idea of how useful it would be to have a keyword that could use a namespace
from an object, hitting the familiar __getattr__ functions and related.

the keyword I proposed would be 'interrogate' (which is rather long, could
use another one like 'using' or something) but.. thats not really important
to me, the idea is, so

class test():
   def __init__(self):
      self.x = 42

foo = test()
bar = test()
x = 33
bar.x = 22
interrogate foo, bar:
  print x

-----

output is 42 since x is found inside of foo before bar, and locals would be
interrogated last, but a more usefull example would be ..

class test():
  def __getattr__(self, name):
    if len(name) == 1 and (ord(name) in range(ord('a'), ord('z'))) :
      return ord(name)


test_ns = test()

interrogate test_ns:
  print a

----
output is 97

using the above example lexical closures may be awkward to implement in the
language itself, this is a bit of a contrived example.. but.. eh..

# if I understand python enough, then I believe that when we enter this
block, and create variables, they go out of scope,
# and there are no exceptions to this rule, ever so .. in order to implement
mine obviously scope would be the same
# mechanism so I have oh_bother declared in the local namespace first

oh_bother = None
interrogate test_ns:
  def geronimo():
    return z
  oh_bother = geronimo

print oh_bother()
---
output is 122

-- 
-Prozacgod

"Prozac may heal the mind, but friends can mend the soul"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090814/db1fc5a5/attachment.html>

From steve at pearwood.info  Fri Aug 14 09:54:03 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 14 Aug 2009 17:54:03 +1000
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
Message-ID: <200908141754.03678.steve@pearwood.info>

On Fri, 14 Aug 2009 04:22:07 pm Prozacgod wrote:
> A while back i had learned about the 'with' statement and at first I
> naively thought that it worked similarly to a with statement I was
> familiar with, from delphi/pascal - obviously it doesn't and I was
> instantly hit with the idea of how useful it would be to have a
> keyword that could use a namespace from an object, hitting the
> familiar __getattr__ functions and related.

What is the motivation? Is it just to reduce typing?

Being an ex-Pascal coder myself, I originally missed having a 
Pascal-like 'with' statement too. But now I'm not so sure it is needed.

I know this proposal (under the name 'with') has been discussed before, 
but I can't find a PEP for it... ah, here you go, it was in the FAQs:

http://www.python.org/doc/faq/general/#why-doesn-t-python-have-a-with-statement-for-attribute-assignments

C-# doesn't have one either:
http://msdn.microsoft.com/en-us/vcsharp/aa336816.aspx

I don't think there are any problems with a Pascal-style 'with' 
statement that couldn't be overcome, but I don't think the benefit is 
great enough to create a new keyword for it. Can you explain in more 
detail why this proposed feature is useful?



-- 
Steven D'Aprano


From steve at pearwood.info  Fri Aug 14 09:58:51 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 14 Aug 2009 17:58:51 +1000
Subject: [Python-ideas] Adding some standalone iterator/sequence
	functions as methods of the iterator objects
In-Reply-To: <10531.1250180588@parc.com>
References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com>
	<42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com>
	<10531.1250180588@parc.com>
Message-ID: <200908141758.51760.steve@pearwood.info>

On Fri, 14 Aug 2009 02:23:08 am Bill Janssen wrote:
> Interesting thread.
>
> Is Python procedural (function-oriented)?  My opinion is that it was,
> when it started out, but isn't really any more, 

Python has always been object-oriented. I don't have direct experience 
with the releases before version 1.5, but all data in Python were 
objects then, and they remain objects now.

Python has *also* always been procedural: Python supports a mix of OO 
and procedural programming very naturally. It would be difficult to 
write a non-trivial program using "pure" OO or "pure" procedural styles 
alone.

Python has also always supported (albeit weakly) a functional style. 
Although it doesn't make the guarantee of "no side-effects" that purely 
functional languages do, Python has provided functional tools like 
filter(), map() and reduce() since the 1.x series, and with more and 
more iterator-based code, Python is arguably becoming more functional 
rather than less.


[...]
> Unfortunately, it seems to me that Python currently makes mixins
> somewhat clumsy to use.  We could use a low-weight way of making a
> class which is a mix of three or four mixins, much as "lambda"
> provides a low-weight way of producing a small function.  Something
> like
>
>     a = class(Mixin1, Mixin2, Mixin3)()
>     a.mixin1_method_foo()
>     a.mixin3_method_bar()

What's wrong with

a.foo()
a.bar()

?

As far as I know, the caller shouldn't need to know or care where a 
inherits the method `foo` from, or even if it is inherited. On the rare 
occasion the caller does want to specify which base class' method to 
call, you can do it with:

Mixin1.foo(a)
Mixin3.bar(a)



-- 
Steven D'Aprano


From ncoghlan at gmail.com  Fri Aug 14 10:11:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 14 Aug 2009 18:11:09 +1000
Subject: [Python-ideas] Interrogate alternate namespace keyword
	and	concept
In-Reply-To: <200908141754.03678.steve@pearwood.info>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<200908141754.03678.steve@pearwood.info>
Message-ID: <4A851C1D.5020507@gmail.com>

Steven D'Aprano wrote:
> I don't think there are any problems with a Pascal-style 'with' 
> statement that couldn't be overcome, but I don't think the benefit is 
> great enough to create a new keyword for it. Can you explain in more 
> detail why this proposed feature is useful?

Also, if you just want to be able to chain multiple namespaces together,
you can do that by implementing an appropriate class with a custom
__getattr__ method.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From ilya.nikokoshev at gmail.com  Fri Aug 14 11:25:53 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Fri, 14 Aug 2009 13:25:53 +0400
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <4A851C1D.5020507@gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<200908141754.03678.steve@pearwood.info> <4A851C1D.5020507@gmail.com>
Message-ID: <d8cc6f9d0908140225u1ba232c2nf7caf71c9b671c2b@mail.gmail.com>

Even if for some reason you needed a statement similar to Pascal's
"with" (and no, I don't know any example where this could be useful
since you'll lose access to anything other then the class), you could
implement it with existing syntax:

class Test:
    '''Class to be interrogated.'''
    def __init__(self, value):
        self.value = value

test = Test(10)

class getattrs(dict):
    '''An auxiliary class.'''
    def __init__(self, instance):
        self.instance = instance
    def __getitem__(self, name):
        return getattr(self.instance, name)
    def __setitem__(self, name, value):
        return setattr(self.instance, name, value)

# interrogate test: value += 5
exec('value += 5', getattrs(test))

print(test.value)
# prints 15.


On Fri, Aug 14, 2009 at 12:11 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
> Steven D'Aprano wrote:
>> I don't think there are any problems with a Pascal-style 'with'
>> statement that couldn't be overcome, but I don't think the benefit is
>> great enough to create a new keyword for it. Can you explain in more
>> detail why this proposed feature is useful?
>
> Also, if you just want to be able to chain multiple namespaces together,
> you can do that by implementing an appropriate class with a custom
> __getattr__ method.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From ilya.nikokoshev at gmail.com  Fri Aug 14 11:35:55 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Fri, 14 Aug 2009 13:35:55 +0400
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <d8cc6f9d0908140225u1ba232c2nf7caf71c9b671c2b@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<200908141754.03678.steve@pearwood.info> <4A851C1D.5020507@gmail.com>
	<d8cc6f9d0908140225u1ba232c2nf7caf71c9b671c2b@mail.gmail.com>
Message-ID: <d8cc6f9d0908140235s722a5e5bk4462d8cf038ef731@mail.gmail.com>

Sorry, it's actually even easier; interrogate() is a one-liner:

class Test:
   '''Class to be interrogated.'''
   def __init__(self, value):
       self.value = value

test = Test(10)

def interrogate(what, how):
   exec(how, what.__dict__)

# interrogate test: value += 5
interrogate(test, 'value += 5')

# interrogate test: print(value)
interrogate(test, 'print(value)')


On Fri, Aug 14, 2009 at 1:25 PM, ilya<ilya.nikokoshev at gmail.com> wrote:
> Even if for some reason you needed a statement similar to Pascal's
> "with" (and no, I don't know any example where this could be useful
> since you'll lose access to anything other then the class), you could
> implement it with existing syntax:
>
> class Test:
> ? ?'''Class to be interrogated.'''
> ? ?def __init__(self, value):
> ? ? ? ?self.value = value
>
> test = Test(10)
>
> class getattrs(dict):
> ? ?'''An auxiliary class.'''
> ? ?def __init__(self, instance):
> ? ? ? ?self.instance = instance
> ? ?def __getitem__(self, name):
> ? ? ? ?return getattr(self.instance, name)
> ? ?def __setitem__(self, name, value):
> ? ? ? ?return setattr(self.instance, name, value)
>
> # interrogate test: value += 5
> exec('value += 5', getattrs(test))
>
> print(test.value)
> # prints 15.
>
>
> On Fri, Aug 14, 2009 at 12:11 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
>> Steven D'Aprano wrote:
>>> I don't think there are any problems with a Pascal-style 'with'
>>> statement that couldn't be overcome, but I don't think the benefit is
>>> great enough to create a new keyword for it. Can you explain in more
>>> detail why this proposed feature is useful?
>>
>> Also, if you just want to be able to chain multiple namespaces together,
>> you can do that by implementing an appropriate class with a custom
>> __getattr__ method.
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
>> ---------------------------------------------------------------
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
>


From steve at pearwood.info  Fri Aug 14 13:17:23 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 14 Aug 2009 21:17:23 +1000
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <d8cc6f9d0908140235s722a5e5bk4462d8cf038ef731@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<d8cc6f9d0908140225u1ba232c2nf7caf71c9b671c2b@mail.gmail.com>
	<d8cc6f9d0908140235s722a5e5bk4462d8cf038ef731@mail.gmail.com>
Message-ID: <200908142117.24072.steve@pearwood.info>

On Fri, 14 Aug 2009 07:35:55 pm ilya wrote:
> Sorry, it's actually even easier; interrogate() is a one-liner:
>
> class Test:
>    '''Class to be interrogated.'''
>    def __init__(self, value):
>        self.value = value
>
> test = Test(10)
>
> def interrogate(what, how):
>    exec(how, what.__dict__)

Apart from the security implications of exec(), it also takes a fairly 
hefty performance hit. In Python 2.6:

>>> from timeit import Timer
>>> setup = 'from __main__ import test, interrogate'
>>> Timer("interrogate(test, 'value += 5')", setup).repeat()
[18.503479957580566, 18.218451023101807, 18.218581914901733]
>>> Timer("test.value += 5", setup).repeat()
[0.33056807518005371, 0.33118104934692383, 0.33114814758300781]

That's a factor of 55 times slower -- not precisely an optimization.



-- 
Steven D'Aprano


From prozacgod at gmail.com  Fri Aug 14 21:25:29 2009
From: prozacgod at gmail.com (Prozacgod)
Date: Fri, 14 Aug 2009 14:25:29 -0500
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <200908141754.03678.steve@pearwood.info>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> 
	<200908141754.03678.steve@pearwood.info>
Message-ID: <90df837e0908141225n2ed9c6f3v68ff3cab55b58e87@mail.gmail.com>

The motivation came up in this thread on python-forum.org

http://www.python-forum.org/pythonforum/viewtopic.php?f=2&t=11606

My main motivation most likely is just that it would open up some very
interesting meta-programing or dynamic programming opportunities, variables
that exists entirely programmatically without the prefix of
object_instance.foo, surely just syntactical sugar to some (or even most) -
I have found it to be valuable syntax.  just like the current with statement
bypasses some syntactical nusances of try: except: .. this would bypass the
nusance of  inst.x inst.y inst.z ...

After posting last night I realized A pitfall of implementing this - would
be

interrogate(foo, bar):
  xylophone = 1024

using this reference - ..
http://docs.python.org/reference/datamodel.html#customizing-attribute-access

__setattr__ has no current way to fall through, so foo would get the setattr
event, and not be able to allow interrogate to hand it to bar.

I posit that raising the AttributeError exception would be sufficient, for
the hand off to occur, and if foo, then bar both did it, then one of two
ways to handle it would be the current scope variables would get it, (eg it
would be __del__ after the end of the block) or putting the new variable in
the parent namespace, since it is somewhat assumed that locals would follow
after bar


On Fri, Aug 14, 2009 at 2:54 AM, Steven D'Aprano <steve at pearwood.info>wrote:

> On Fri, 14 Aug 2009 04:22:07 pm Prozacgod wrote:
> > A while back i had learned about the 'with' statement and at first I
> > naively thought that it worked similarly to a with statement I was
> > familiar with, from delphi/pascal - obviously it doesn't and I was
> > instantly hit with the idea of how useful it would be to have a
> > keyword that could use a namespace from an object, hitting the
> > familiar __getattr__ functions and related.
>
> What is the motivation? Is it just to reduce typing?
>
> Being an ex-Pascal coder myself, I originally missed having a
> Pascal-like 'with' statement too. But now I'm not so sure it is needed.
>
> I know this proposal (under the name 'with') has been discussed before,
> but I can't find a PEP for it... ah, here you go, it was in the FAQs:
>
>
> http://www.python.org/doc/faq/general/#why-doesn-t-python-have-a-with-statement-for-attribute-assignments
>
> C-# doesn't have one either:
> http://msdn.microsoft.com/en-us/vcsharp/aa336816.aspx
>
> I don't think there are any problems with a Pascal-style 'with'
> statement that couldn't be overcome, but I don't think the benefit is
> great enough to create a new keyword for it. Can you explain in more
> detail why this proposed feature is useful?
>
>
>
> --
> Steven D'Aprano
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
-Prozacgod

"Prozac may heal the mind, but friends can mend the soul"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090814/04c99b82/attachment.html>

From prozacgod at gmail.com  Fri Aug 14 21:46:17 2009
From: prozacgod at gmail.com (Prozacgod)
Date: Fri, 14 Aug 2009 14:46:17 -0500
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
Message-ID: <90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com>

I just read this inside of the C# reasoning, and this pretty much kinda
says, look its to difficult...

This is a good reason "Another approach is to push a scope and make the
property hide the local variable, but then there's no way to refer to the
local without adding some escape syntax."

bascially, they added the dot syntax to acces variables that are intended to
be directed towards object namespace.  Which I do agree helps with
disambiguation,

foo = 32
interrogate(test_ns):
  print .foo
  print foo

But don't find all that paletable.  My thoughts on this would be to cover
the parent local namespace, and even globals with your object, the object
gets it first everytime, and if it raised an exception, then the normal
method would be implemented.  If it were important to have a variable that
was named the same, then access it outide the interrogate block or
otherwise..  But if you were a developer and working on writing a program,
knowing you wanted to use interrogate, then wouldn't you be wise to choose
different variable names?  I think I would.

Also doing this is kinda kludgy feeling to me..

function(args).dict[index][index].a = 21
function(args).dict[index][index].b = 42
function(args).dict[index][index].c = 63

write this:

ref = function(args).dict[index][index]
ref.a = 21
ref.b = 42
ref.c = 63

even though it is optimised, and works well, this feels more like the
programmer adapting to the environment instead of the environment working
for the programmer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090814/f4bf7992/attachment.html>

From ilya.nikokoshev at gmail.com  Fri Aug 14 22:15:19 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Sat, 15 Aug 2009 00:15:19 +0400
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com>
Message-ID: <d8cc6f9d0908141315w6cbe0b8di36986a1eaf7e77e9@mail.gmail.com>

Well,

> foo = 32
> interrogate(test_ns):
>   print .foo
>   print foo
>

can be changed to

foo = 32
_ = test_ns
print _.foo
print foo

which actually has less symbols and doesn't have any ambiguity as for
the meaning of the new statement!

> But don't find all that paletable.  My thoughts on this would be to cover
> the parent local namespace, and even globals with your object, the object

If a suggestion is to resolve names into several dictionaries, I don't
think this is practical in any way! As the source you cited correctly
explains, there is an irreparable problem. Suppose I write

foo = 32
interrogate(test_ns):
   print bar  # test_ns.bar
   print foo  # global foo

Then some time later I add global variable bar somewhere in the module
and the program is mysteriously broken! The same thing happens if you
first consult test_ns.__dict__ -- somebody can always add foo to
test_ns, its parent or *even object class*.

So I don't think there's a good way to define semantics of
'interrogate' without some kind of dot-syntax, which is debated for
Python for quite a long time without success.


On Fri, Aug 14, 2009 at 11:46 PM, Prozacgod<prozacgod at gmail.com> wrote:
> I just read this inside of the C# reasoning, and this pretty much kinda
> says, look its to difficult...
>
> This is a good reason "Another approach is to push a scope and make the
> property hide the local variable, but then there's no way to refer to the
> local without adding some escape syntax."
>
> bascially, they added the dot syntax to acces variables that are intended to
> be directed towards object namespace.? Which I do agree helps with
> disambiguation,
>
> foo = 32
> interrogate(test_ns):
> ? print .foo
> ? print foo
>
> But don't find all that paletable.? My thoughts on this would be to cover
> the parent local namespace, and even globals with your object, the object
> gets it first everytime, and if it raised an exception, then the normal
> method would be implemented.? If it were important to have a variable that
> was named the same, then access it outide the interrogate block or
> otherwise..? But if you were a developer and working on writing a program,
> knowing you wanted to use interrogate, then wouldn't you be wise to choose
> different variable names?? I think I would.
>
> Also doing this is kinda kludgy feeling to me..
>
> function(args).dict[index][index].a = 21
> function(args).dict[index][index].b = 42
> function(args).dict[index][index].c = 63
>
> write this:
>
> ref = function(args).dict[index][index]
> ref.a = 21
> ref.b = 42
> ref.c = 63
>
> even though it is optimised, and works well, this feels more like the
> programmer adapting to the environment instead of the environment working
> for the programmer.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


From ilya.nikokoshev at gmail.com  Fri Aug 14 22:29:11 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Sat, 15 Aug 2009 00:29:11 +0400
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <200908142117.24072.steve@pearwood.info>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<d8cc6f9d0908140225u1ba232c2nf7caf71c9b671c2b@mail.gmail.com>
	<d8cc6f9d0908140235s722a5e5bk4462d8cf038ef731@mail.gmail.com>
	<200908142117.24072.steve@pearwood.info>
Message-ID: <d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>

That's a useful statistics, but the bottleneck is **only** because of
parsing 'value +=5'.

Here's how I time it:

# insert my old program here...


from timeit import Timer
from codeop import compile_command

def timing(something):
    setup = 'from __main__ import test, interrogate, command, inc5'
    best = sorted(Timer(something, setup).repeat(3, 1000))[0]
    print('{0!r} -> {1:.3} ms'.format(something, best))
    print('# test.value =', test.value)

command = '''
for i in range(10):
    value += 1
'''

inc5 = compile_command(command)

timing("interrogate(test, command)")
timing(command.replace('value', 'test.value'))
timing("interrogate(test, inc5)")

Result:

15
'interrogate(test, command)' -> 0.0908 ms
# test.value = 30015
'\nfor i in range(10):\n    test.value += 1\n' -> 0.00408 ms
# test.value = 60015
'interrogate(test, inc5)' -> 0.00469 ms
# test.value = 90015

so interrogate() with additional precompiling introduces very little
overhead. Though I agree it's inconvenient to write functions as
strings; I think someone smarter than me can find a way to do it like
a regular function call.



On Fri, Aug 14, 2009 at 3:17 PM, Steven D'Aprano<steve at pearwood.info> wrote:
> On Fri, 14 Aug 2009 07:35:55 pm ilya wrote:
>> Sorry, it's actually even easier; interrogate() is a one-liner:
>>
>> class Test:
>> ? ?'''Class to be interrogated.'''
>> ? ?def __init__(self, value):
>> ? ? ? ?self.value = value
>>
>> test = Test(10)
>>
>> def interrogate(what, how):
>> ? ?exec(how, what.__dict__)
>
> Apart from the security implications of exec(), it also takes a fairly
> hefty performance hit. In Python 2.6:
>
>>>> from timeit import Timer
>>>> setup = 'from __main__ import test, interrogate'
>>>> Timer("interrogate(test, 'value += 5')", setup).repeat()
> [18.503479957580566, 18.218451023101807, 18.218581914901733]
>>>> Timer("test.value += 5", setup).repeat()
> [0.33056807518005371, 0.33118104934692383, 0.33114814758300781]
>
> That's a factor of 55 times slower -- not precisely an optimization.
>
>
>
> --
> Steven D'Aprano
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From prozacgod at gmail.com  Sat Aug 15 01:53:41 2009
From: prozacgod at gmail.com (Prozacgod)
Date: Fri, 14 Aug 2009 18:53:41 -0500
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> 
	<d8cc6f9d0908140225u1ba232c2nf7caf71c9b671c2b@mail.gmail.com> 
	<d8cc6f9d0908140235s722a5e5bk4462d8cf038ef731@mail.gmail.com> 
	<200908142117.24072.steve@pearwood.info>
	<d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>
Message-ID: <90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com>

This makes sense, obviously you should compile it first, which when executed
should execute exactly the same as the original context python.

On Fri, Aug 14, 2009 at 3:29 PM, ilya <ilya.nikokoshev at gmail.com> wrote:

> That's a useful statistics, but the bottleneck is **only** because of
> parsing 'value +=5'.
>
> Here's how I time it:
>
> # insert my old program here...
>
>
> from timeit import Timer
> from codeop import compile_command
>
> def timing(something):
>    setup = 'from __main__ import test, interrogate, command, inc5'
>    best = sorted(Timer(something, setup).repeat(3, 1000))[0]
>    print('{0!r} -> {1:.3} ms'.format(something, best))
>    print('# test.value =', test.value)
>
> command = '''
> for i in range(10):
>    value += 1
> '''
>
> inc5 = compile_command(command)
>
> timing("interrogate(test, command)")
> timing(command.replace('value', 'test.value'))
> timing("interrogate(test, inc5)")
>
> Result:
>
> 15
> 'interrogate(test, command)' -> 0.0908 ms
> # test.value = 30015
> '\nfor i in range(10):\n    test.value += 1\n' -> 0.00408 ms
> # test.value = 60015
> 'interrogate(test, inc5)' -> 0.00469 ms
> # test.value = 90015
>
> so interrogate() with additional precompiling introduces very little
> overhead. Though I agree it's inconvenient to write functions as
> strings; I think someone smarter than me can find a way to do it like
> a regular function call.
>
>
>
> On Fri, Aug 14, 2009 at 3:17 PM, Steven D'Aprano<steve at pearwood.info>
> wrote:
> > On Fri, 14 Aug 2009 07:35:55 pm ilya wrote:
> >> Sorry, it's actually even easier; interrogate() is a one-liner:
> >>
> >> class Test:
> >>    '''Class to be interrogated.'''
> >>    def __init__(self, value):
> >>        self.value = value
> >>
> >> test = Test(10)
> >>
> >> def interrogate(what, how):
> >>    exec(how, what.__dict__)
> >
> > Apart from the security implications of exec(), it also takes a fairly
> > hefty performance hit. In Python 2.6:
> >
> >>>> from timeit import Timer
> >>>> setup = 'from __main__ import test, interrogate'
> >>>> Timer("interrogate(test, 'value += 5')", setup).repeat()
> > [18.503479957580566, 18.218451023101807, 18.218581914901733]
> >>>> Timer("test.value += 5", setup).repeat()
> > [0.33056807518005371, 0.33118104934692383, 0.33114814758300781]
> >
> > That's a factor of 55 times slower -- not precisely an optimization.
> >
> >
> >
> > --
> > Steven D'Aprano
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > http://mail.python.org/mailman/listinfo/python-ideas
> >
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
-Prozacgod

"Prozac may heal the mind, but friends can mend the soul"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090814/29c8ee72/attachment.html>

From prozacgod at gmail.com  Sat Aug 15 01:59:11 2009
From: prozacgod at gmail.com (Prozacgod)
Date: Fri, 14 Aug 2009 18:59:11 -0500
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <d8cc6f9d0908141315w6cbe0b8di36986a1eaf7e77e9@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> 
	<90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com> 
	<d8cc6f9d0908141315w6cbe0b8di36986a1eaf7e77e9@mail.gmail.com>
Message-ID: <90df837e0908141659k32dffb8fod69bf647736193f4@mail.gmail.com>

Oh .. good call, you right, this would be difficult to observe, find, and
correct

>foo = 32
>interrogate(test_ns):
>  print bar  # test_ns.bar
>  print foo  # global foo

>Then some time later I add global variable bar somewhere in the module
>and the program is mysteriously broken! The same thing happens if you
>first consult test_ns.__dict__ -- somebody can always add foo to
>test_ns, its parent or *even object class*.

In delphi/pascal they left this up to the developer, so you're supposed to
divine that a new property was made on a class, to me that is a reasonable
trade-off, I could choose to only use this syntax on code that I'm writing
and supporting, or realize something could break - BUT I also understand the
mentality of - it shouldn't randomly break ever, which is a reasonable
supposition of code you write.

After this discussion I'm wondering why it ever ended up in any language to
begin with?  Seems like it opens up to many issues to the developers using
the language and the language implementors.

On Fri, Aug 14, 2009 at 3:15 PM, ilya <ilya.nikokoshev at gmail.com> wrote:

> Well,
>
> > foo = 32
> > interrogate(test_ns):
> >   print .foo
> >   print foo
> >
>
> can be changed to
>
> foo = 32
> _ = test_ns
> print _.foo
> print foo
>
> which actually has less symbols and doesn't have any ambiguity as for
> the meaning of the new statement!
>
> > But don't find all that paletable.  My thoughts on this would be to cover
> > the parent local namespace, and even globals with your object, the object
>
> If a suggestion is to resolve names into several dictionaries, I don't
> think this is practical in any way! As the source you cited correctly
> explains, there is an irreparable problem. Suppose I write
>
> foo = 32
> interrogate(test_ns):
>    print bar  # test_ns.bar
>   print foo  # global foo
>
> Then some time later I add global variable bar somewhere in the module
> and the program is mysteriously broken! The same thing happens if you
> first consult test_ns.__dict__ -- somebody can always add foo to
> test_ns, its parent or *even object class*.
>
> So I don't think there's a good way to define semantics of
> 'interrogate' without some kind of dot-syntax, which is debated for
> Python for quite a long time without success.
>
>
> On Fri, Aug 14, 2009 at 11:46 PM, Prozacgod<prozacgod at gmail.com> wrote:
> > I just read this inside of the C# reasoning, and this pretty much kinda
> > says, look its to difficult...
> >
> > This is a good reason "Another approach is to push a scope and make the
> > property hide the local variable, but then there's no way to refer to the
> > local without adding some escape syntax."
> >
> > bascially, they added the dot syntax to acces variables that are intended
> to
> > be directed towards object namespace.  Which I do agree helps with
> > disambiguation,
> >
> > foo = 32
> > interrogate(test_ns):
> >   print .foo
> >   print foo
> >
> > But don't find all that paletable.  My thoughts on this would be to cover
> > the parent local namespace, and even globals with your object, the object
> > gets it first everytime, and if it raised an exception, then the normal
> > method would be implemented.  If it were important to have a variable
> that
> > was named the same, then access it outide the interrogate block or
> > otherwise..  But if you were a developer and working on writing a
> program,
> > knowing you wanted to use interrogate, then wouldn't you be wise to
> choose
> > different variable names?  I think I would.
> >
> > Also doing this is kinda kludgy feeling to me..
> >
> > function(args).dict[index][index].a = 21
> > function(args).dict[index][index].b = 42
> > function(args).dict[index][index].c = 63
> >
> > write this:
> >
> > ref = function(args).dict[index][index]
> > ref.a = 21
> > ref.b = 42
> > ref.c = 63
> >
> > even though it is optimised, and works well, this feels more like the
> > programmer adapting to the environment instead of the environment working
> > for the programmer.
> >
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > http://mail.python.org/mailman/listinfo/python-ideas
> >
> >
>



-- 
-Prozacgod

"Prozac may heal the mind, but friends can mend the soul"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090814/3dca925e/attachment.html>

From steve at pearwood.info  Sat Aug 15 04:02:29 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 15 Aug 2009 12:02:29 +1000
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<200908142117.24072.steve@pearwood.info>
	<d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>
Message-ID: <200908151202.29948.steve@pearwood.info>

On Sat, 15 Aug 2009 06:29:11 am ilya wrote:
> That's a useful statistics, but the bottleneck is **only** because of
> parsing 'value +=5'.
>
> Here's how I time it:
>
> # insert my old program here...
>
>
> from timeit import Timer
> from codeop import compile_command
>
> def timing(something):
>     setup = 'from __main__ import test, interrogate, command, inc5'
>     best = sorted(Timer(something, setup).repeat(3, 1000))[0]

min(alist) is a more-direct, simpler, faster, easier to read way of 
calculating sorted(alist)[0].


>     print('{0!r} -> {1:.3} ms'.format(something, best))
>     print('# test.value =', test.value)
>
> command = '''
> for i in range(10):
>     value += 1
> '''

You're muddying the water by including a for-loop and call to range() 
inside the code snippet being tested. We're trying to compare your 
function interrogate(test, 'value += 1') with the standard call to 
test.value += 1. Why include the time required to generate a range() 
object, and iterate over it ten times, as part of the code snippet? All 
that does is mix up the time required to execute common code and the 
time required to execute the code we care about.

If it's not obvious why your approach is flawed, consider this extreme 
example:

Timer("time.sleep(1000); interrogate(test, 'value += 1')", ...)
Timer("time.sleep(1000); test.value += 1", ...)

The differences in speed between the interrogate call (using exec) and 
the direct access to test.value will be swamped by the time used by the 
common code. 

A more accurate measurement is to remove the "for i in..." part from 
command, and increase the number=1000 argument to Timer.repeat() to 
10000.


> inc5 = compile_command(command)

This is an unfair test. We're comparing directly accessing test.value 
versus indirectly accessing test.value using exec. Regardless of 
whether the caller compiles the statement manually before passing it to 
exec, or just passes it to exec to compile it automatically, the cost 
of that compilation has to be payed. Pulling that outside of the timing 
code just hides the true cost.

Pre-compiling before passing to exec is a good optimization for the 
cases where you need to exec the same code snippet over and over again. 
If you want to execute "for i in range(1000): exec(s)" then it makes 
sense to pull out the compilation of s outside of the loop. But 
generally when timing code snippets, the only purpose of the loop is to 
minimize errors and give more accurate results, so pulling out the 
compilation just hides some of the real cost.


> timing("interrogate(test, command)")
> timing(command.replace('value', 'test.value'))
> timing("interrogate(test, inc5)")
>
> Result:
>
> 15

Where does the 15 come from?

> 'interrogate(test, command)' -> 0.0908 ms
> # test.value = 30015
> '\nfor i in range(10):\n    test.value += 1\n' -> 0.00408 ms
> # test.value = 60015
> 'interrogate(test, inc5)' -> 0.00469 ms
> # test.value = 90015
>
> so interrogate() with additional precompiling introduces very little
> overhead. 

Only because you're ignoring the overhead of pre-compiling. A more 
accurate test would be:

timing("inc5 = compile_command(command); interrogate(test, inc5)")


> Though I agree it's inconvenient to write functions as 
> strings; 

If you think that's inconvenient, just try writing functions as code 
objects without calling compile :)



-- 
Steven D'Aprano


From steve at pearwood.info  Sat Aug 15 04:13:22 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 15 Aug 2009 12:13:22 +1000
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>
	<90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com>
Message-ID: <200908151213.23078.steve@pearwood.info>

On Sat, 15 Aug 2009 09:53:41 am Prozacgod wrote:
> This makes sense, obviously you should compile it first, which when
> executed should execute exactly the same as the original context
> python.

We want to compare the speed of the following statement:

test.value += 1

with the equivalent:

code = compile("value += 1", '', 'exec')
exec code in test.__dict__


Why do you think it is appropriate to ignore the time taken by the 
compile() when comparing the two? Put it this way: if you want to know 
how long it takes to make fresh stir-fried chicken, you have to include 
the ten minutes it takes to chop up the vegetables and chicken, and not 
just the two minutes it takes to stir-fry them.



-- 
Steven D'Aprano


From solipsis at pitrou.net  Sat Aug 15 13:19:37 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Aug 2009 11:19:37 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?q?Interrogate_alternate_namespace_keyword_?=
	=?utf-8?q?and=09concept?=
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<200908141754.03678.steve@pearwood.info>
	<90df837e0908141225n2ed9c6f3v68ff3cab55b58e87@mail.gmail.com>
Message-ID: <loom.20090815T111551-711@post.gmane.org>

Prozacgod <prozacgod at ...> writes:
> 
> My main motivation most likely is just that it would open up some very 
> interesting meta-programing or dynamic programming opportunities, variables 
> that exists entirely programmatically without the prefix of 
> object_instance.foo, surely just syntactical sugar to some (or even most) -

This sounds very PHP-like (doing indirections like $$foo because there isn't any
powerful introspection or OO system, and many things (e.g. functions, classes)
aren't first-class objects and can't be passed as parameters). I don't think
Python should grow something like that; instead people should learn more
structured ways of accessing data. getattr() and friends are powerful enough for
playing access tricks.

(in other words: -1 from me)




From dickinsm at gmail.com  Sat Aug 15 13:23:17 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 15 Aug 2009 12:23:17 +0100
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <4A801D0C.2000502@gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
Message-ID: <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>

On Mon, Aug 10, 2009 at 2:13 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
> Bytes to int:
>
> [...]
> Int to bytes:
>
> [...]

Alexandre Vassalotti has posted a patch at

http://bugs.python.org/issue1023290

that implements methods very much like the ones that Nick describes.

Mark


From ede at mit.edu  Sat Aug 15 14:13:51 2009
From: ede at mit.edu (Eric Eisner)
Date: Sat, 15 Aug 2009 21:13:51 +0900
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>
Message-ID: <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>

This patch seems very complete, with only the API to hammered out.
Here is my summary of some options:

method names:
patch behavior: int.as_bytes / int.frombytes
int.as_bytes / int.from_bytes
int.asbytes / int.frombytes

Endianness:
patch behavior: default flag: little_endian=False
byteorder option accepting 'big' or 'little', this can also accept sys.byteorder

sign:
patch behavior: default flag: signed=True
maybe unsigned as the default?

byte length:
patch behavior: fixed_length=None
other names: length, bytelength


As for my own opinions: I think the method names should have
consistent underscore usage. I think it is important to have all of
big, little, and native as byteorder options, but I would be against
having native as the default. I think it is unclean for core
functionality to be platform dependent (there may be some examples of
this that I'm not thinking of though).

One option would be to have the defaults of int.as_bytes mirror the
hex builtin function, eg big endian and unsigned. This way it could be
more consistent with related functionality already in the core.

Thoughts?

-Eric

On Sat, Aug 15, 2009 at 20:23, Mark Dickinson<dickinsm at gmail.com> wrote:
> On Mon, Aug 10, 2009 at 2:13 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
>> Bytes to int:
>>
>> [...]
>> Int to bytes:
>>
>> [...]
>
> Alexandre Vassalotti has posted a patch at
>
> http://bugs.python.org/issue1023290
>
> that implements methods very much like the ones that Nick describes.
>
> Mark
>


From dickinsm at gmail.com  Sat Aug 15 14:14:40 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 15 Aug 2009 13:14:40 +0100
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <4A801D0C.2000502@gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
Message-ID: <5c6f2a5d0908150514l31d3532an382d6372db30c410@mail.gmail.com>

(I'm repeating some of the comments already made in the bug-tracker;
as Antoine pointed out, discussion should probably remain here
until the API is settled.)

On Mon, Aug 10, 2009 at 2:13 PM, Nick Coghlan<ncoghlan at gmail.com> wrote:
> [...]
>
> That suggests to me the following signatures for the conversion
> functions (regardless of the names they might be given):
>
> int.from_bytes(data, *, little_endian=None, signed=True)
> ?little_endian would become a three-valued parameter for the Python
> version: ?None = native; False = little-endian; True = big-endian. The
> three valued parameter is necessary since Python code can't get at the
> "IS_LITTLE_ENDIAN" macro that the PyLong code uses to determine the
> endianness of the system when calling the C API functions.
> ?signed would just be an ordinary boolean flag

Sounds good to me.  I'm not sure about the 'signed=True' default;
to me, a default of unsigned seems more natural.  But this is
bikeshedding, and I'd happily accept either default.

I agree with other posters that there seems little reason not
to accept the empty string.  It's a natural end-case for unsigned
input; whether it's natural for signed input (where there should
really be at least one 'sign bit', and hence at least one byte)
is arguable, but I can't see the harm in accepting it.

> int.to_bytes(data, *, size=0, little_endian=None, signed=True)
> ?A size <= 0 would mean to produce as many bytes as are needed to
> represent the integer and no more. Otherwise it would represent the
> maximum number of bytes allowed in the response (raising OverflowError
> if the value won't fit).
> ?little_endian and signed would be interpreted as for the conversion
> from bytes to an integer

I'm not convinced that it's valuable to a have a variable-size
version of this;  I'd make size a required argument.

The problem with the variable size version is that the choice of
byte-length for the output for a given integer input is a little bit
arbitrary.  For a particular requirement (producing code to conform
with some existing serialization protocol, for example) it seems
likely that the choice Python makes will disagree with what's
required by that protocol, so that size still has to be given explicitly.
On the other hand, if a user just wants a quick and easy way
to serialize ints, without caring about the exact form of the
serialization, then there are number of solutions already
available within Python.

+1 on raising OverflowError for out-of-range inputs, instead of
wrapping modulo 2**whatever.  This also fits with the way that
the struct module currently behaves.

Does anyone see other use-cases for variable-size conversion?

[Greg Ewing]
> I don't like the idea of a three-valued boolean. I also
> don't like boolean parameters whose sense is abritrary
> (why is it called "little_endian" and not "big_endian",
> and how do I remember which convention was chosen?)

> My suggestion would be to use the same characters that
> the struct module uses to represent endianness (">"
> for big-endian, "<" for little-endian, etc.)

How about a parameter byteorder=None, accepting values
'big' and 'little'?  Then one could use byteorder=sys.byteorder
to explicitly specify native byteorder.


Mark


From ncoghlan at gmail.com  Sat Aug 15 17:08:26 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Aug 2009 01:08:26 +1000
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com>	
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>	
	<200908081554.36358.steve@pearwood.info>	
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>	
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>	
	<4A7FF6CE.7070903@gmail.com>	
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>	
	<4A801D0C.2000502@gmail.com>	
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
Message-ID: <4A86CF6A.7070409@gmail.com>

Eric Eisner wrote:
> One option would be to have the defaults of int.as_bytes mirror the
> hex builtin function, eg big endian and unsigned. This way it could be
> more consistent with related functionality already in the core.

Having the byte conversion mirror the hex builtin is probably the least
arbitrary style guide we could come up with for the defaults.

The other option would be to not *have* defaults for either of these
settings and force the programmer to make a deliberate choice. Then once
it has been out in the field for a while and we have evidence about the
way people use it, add sensible defaults in the following release.

I would prefer either of those two options to attempting to just guess
what the appropriate defaults would be in the absence of a wide
selection of use cases.

I also like the idea of using sys.byteorder as the model for the
byteorder selection option (i.e. byteorder=sys.byteorder to select
native layout and 'big'/'little' to choose a specific one)

I think Mark also makes a good point that in cases where the size
doesn't matter, marshal or pickle is probably a better choice than this
direct conversion so +0 on requiring an explicit size on the conversion
to bytes. Again, this is a case where *adding* a default later would be
easy, but changing a default or removing a variable size feature would
be difficult.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From alexandre at peadrop.com  Sat Aug 15 17:45:37 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Sat, 15 Aug 2009 11:45:37 -0400
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> 
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com> 
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> 
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> 
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> 
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
Message-ID: <acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>

[Oops, I forgot to CC python-ideas. Sorry Eric for the double post.]

On Sat, Aug 15, 2009 at 8:13 AM, Eric Eisner<ede at mit.edu> wrote:
> This patch seems very complete, with only the API to hammered out.
> Here is my summary of some options:
>
> method names:
> patch behavior: int.as_bytes / int.frombytes
> int.as_bytes / int.from_bytes
> int.asbytes / int.frombytes
>

In my patch, I chosen to use the name 'as_bytes' because it was
consistent with 'float.as_integer_ratio'. Similarly, I chosen the name
'frombytes'  because it was consistent with 'float.fromhex'. I don't
mind the inconsistent use of the underscore in the names, but I admit
there is room for improvement. So, what do you think of
`int.frombytes` and `int.tobytes`?

> Endianness:
> patch behavior: default flag: little_endian=False
> byteorder option accepting 'big' or 'little', this can also accept sys.byteorder
>

I like the byteorder option better.

I believe the byteorder option shouldn't default to use the native
byte-order however. As you mentioned, it would be a bad choice to
encourage the default behaviour to be platform-dependent. And since
the primary purpose of the API is long serialization, it would be
short-sighted to choose the option that cannot be used for
serialization as the default.

Whether it should default to 'little' or 'big' is pretty much an
arbitrary choice. In my patch, I choose to default big-endian since it
is the standard network byte-order. But maybe the option should
default to little-endian instead since it more widely used. In
addition, the patch is slightly more efficient with little-endian.

> sign:
> patch behavior: default flag: signed=True
> maybe unsigned as the default?
>

Either is fine by me. The advantage with 'signed' as the default is
'signed' works with all longs (and not only with non-negative ones).

> byte length:
> patch behavior: fixed_length=None
> other names: length, bytelength
>

I still like `fixed_length` better than proposed alternatives. The
name `fixed_length` makes it clear that the returned object has a
fixed and constant length. And, I find `fixed_length=None` is more
telling than `length=None`.

-- Alexandre


From dickinsm at gmail.com  Sat Aug 15 18:25:02 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 15 Aug 2009 17:25:02 +0100
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>
Message-ID: <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com>

On Sat, Aug 15, 2009 at 4:45 PM, Alexandre
Vassalotti<alexandre at peadrop.com> wrote:
> I believe the byteorder option shouldn't default to use the native
> byte-order however. As you mentioned, it would be a bad choice to
> encourage the default behaviour to be platform-dependent. And since
> the primary purpose of the API is long serialization, it would be
> short-sighted to choose the option that cannot be used for
> serialization as the default.
>
> Whether it should default to 'little' or 'big' is pretty much an
> arbitrary choice. In my patch, I choose to default big-endian since it
> is the standard network byte-order. But maybe the option should
> default to little-endian instead since it more widely used. In
> addition, the patch is slightly more efficient with little-endian.

Given all this, it sounds like byteorder should be a required argument.
'In the face of ambiguity... ' and all that.  As Nick pointed out, we
can always add a default later when a larger set of use-cases has
emerged.

You say that the 'primary purpose of the API is long serialization'.
I'd argue that that's not quite true.  That is, I see two separate
uses:

(1) fixed-size conversions:  e.g., interpreting a three-byte sequence
as an integer for the purposes of bit operations, or converting
an int generated by random.getrandbits(k) to a random byte sequence.
(See http://bugs.python.org/msg69285 and
http://bugs.python.org/msg54262.)

(2) Provide a primitive operation that's useful for serialization
protocols.  Here I'd guess that the details of e.g., how a negative
integer is serialized would vary from protocol to protocol, so
that the serialization code would in most cases still end up
having to specify the fixed_length argument.

I *don't* see int.tobytes and int.frombytes (or whatever the names
turn out to be) as providing integer serialization by themselves.
There's no need for this, since pickle and marshal already do
this job.  Incidentally, the commenters in

http://bugs.python.org/issue467384

have quite a lot to say on this subject.  It's on this basis that
I'm suggesting that the size argument should be required for
int.tobytes.

> On Sat, Aug 15, 2009 at 8:13 AM, Eric Eisner<ede at mit.edu> wrote:
> > sign:
> > patch behavior: default flag: signed=True
> > maybe unsigned as the default?
> Either is fine by me. The advantage with 'signed' as the default is
> 'signed' works with all longs (and not only with non-negative ones).

Agreed.  Of course, this advantage disappears if the size argument
is mandatory.

I don't have any strong opinions about the method and parameter
names, so I'll keep quiet on that subject. :)

Mark


From alexandre at peadrop.com  Sat Aug 15 18:34:47 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Sat, 15 Aug 2009 12:34:47 -0400
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <5c6f2a5d0908150514l31d3532an382d6372db30c410@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> 
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> 
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com> 
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> 
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> 
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150514l31d3532an382d6372db30c410@mail.gmail.com>
Message-ID: <acd65fa20908150934q4bdb32cfq7c2d599e10f2685f@mail.gmail.com>

On Sat, Aug 15, 2009 at 8:14 AM, Mark Dickinson<dickinsm at gmail.com> wrote:
> I'm not convinced that it's valuable to a have a variable-size
> version of this; ?I'd make size a required argument.
>
> The problem with the variable size version is that the choice of
> byte-length for the output for a given integer input is a little bit
> arbitrary. ?For a particular requirement (producing code to conform
> with some existing serialization protocol, for example) it seems
> likely that the choice Python makes will disagree with what's
> required by that protocol, so that size still has to be given explicitly.
> On the other hand, if a user just wants a quick and easy way
> to serialize ints, without caring about the exact form of the
> serialization, then there are number of solutions already
> available within Python.
>

Well, the only use-case in the standard library I found (i.e.,
simplifying encode_long() and decode_long() in pickle.py) needed the
variable-length version. However, unlike I originally thought, the
variable length version is not difficult to emulate using
`int.bit_length()`. For example, with my patch I can rewrite:

def encode_long(x):
    if x == 0:
        return b""
    return x.as_bytes(little_endian=True)

as:

def encode_long(x)
    if x == 0:
        return b""
    nbytes = (x.bit_length() >> 3) + 1
    result = x.as_bytes(nbytes, little_endian=True)
    if x < 0 and nbytes > 1:
        if result[-1] == 0xff and (result[-2] & 0x80) != 0:
            result = result[:-1]
    return result

I usually hate with passion APIs that requires you to know the length
of the result in advance. But this doesn't look bad. The only use-case
for the variable-length version I have is the encode_long() function
in pickle.py. In addition, it sounds reasonable to leave the duty of
long serialization to pickle.

So, +1 from me.

-- Alexandre


From alexandre at peadrop.com  Sat Aug 15 18:48:12 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Sat, 15 Aug 2009 12:48:12 -0400
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com> 
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> 
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> 
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> 
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> 
	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com> 
	<5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com>
Message-ID: <acd65fa20908150948k403c52d0t99b80ae20656546b@mail.gmail.com>

On Sat, Aug 15, 2009 at 12:25 PM, Mark Dickinson<dickinsm at gmail.com> wrote:
> On Sat, Aug 15, 2009 at 4:45 PM, Alexandre
> Vassalotti<alexandre at peadrop.com> wrote:
>> I believe the byteorder option shouldn't default to use the native
>> byte-order however. As you mentioned, it would be a bad choice to
>> encourage the default behaviour to be platform-dependent. And since
>> the primary purpose of the API is long serialization, it would be
>> short-sighted to choose the option that cannot be used for
>> serialization as the default.
>>
>> Whether it should default to 'little' or 'big' is pretty much an
>> arbitrary choice. In my patch, I choose to default big-endian since it
>> is the standard network byte-order. But maybe the option should
>> default to little-endian instead since it more widely used. In
>> addition, the patch is slightly more efficient with little-endian.
>
> Given all this, it sounds like byteorder should be a required argument.
> 'In the face of ambiguity... ' and all that. ?As Nick pointed out, we
> can always add a default later when a larger set of use-cases has
> emerged.
>
> You say that the 'primary purpose of the API is long serialization'.
> I'd argue that that's not quite true.

And you are totally right. Honestly, the only reason I was thinking
about long serialization is because I have my hand full with pickle
guts presently. :-)

-- Alexandre


From prozacgod at gmail.com  Sat Aug 15 19:57:58 2009
From: prozacgod at gmail.com (Prozacgod)
Date: Sat, 15 Aug 2009 12:57:58 -0500
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <loom.20090815T111551-711@post.gmane.org>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> 
	<200908141754.03678.steve@pearwood.info>
	<90df837e0908141225n2ed9c6f3v68ff3cab55b58e87@mail.gmail.com> 
	<loom.20090815T111551-711@post.gmane.org>
Message-ID: <90df837e0908151057g4096advee80bac742cb1649@mail.gmail.com>

I'm leaning towards a -0.5 right now myself, simply because of the
ambiguities that have been mentioned, python is anything but ambiguous, and
that's a distinction I like about it, in every case I can think of python
does what you'd expect without random oh, but there is this qualifier in
this context .. (*cough* perl *cough* ruby *cough*).. even if I think I like
the syntax, it doesn't really make sense to implement it at this time, or
likely ever.

I got the inspiration for the idea from delphi/pascal, but in retrospect it
turns out that the primary large (~1million lines) commercial app I worked
on where we used that syntax (heavily I might add), they started to phase
out the use of the with statement all over the code because of these same
abiguities creating obscure difficult to track down bugs.

On Sat, Aug 15, 2009 at 6:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Prozacgod <prozacgod at ...> writes:
> >
> > My main motivation most likely is just that it would open up some very
> > interesting meta-programing or dynamic programming opportunities,
> variables
> > that exists entirely programmatically without the prefix of
> > object_instance.foo, surely just syntactical sugar to some (or even most)
> -
>
> This sounds very PHP-like (doing indirections like $$foo because there
> isn't any
> powerful introspection or OO system, and many things (e.g. functions,
> classes)
> aren't first-class objects and can't be passed as parameters). I don't
> think
> Python should grow something like that; instead people should learn more
> structured ways of accessing data. getattr() and friends are powerful
> enough for
> playing access tricks.
>
> (in other words: -1 from me)
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
-Prozacgod

"Prozac may heal the mind, but friends can mend the soul"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090815/8641f7d0/attachment.html>

From dickinsm at gmail.com  Sat Aug 15 20:42:23 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 15 Aug 2009 19:42:23 +0100
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <acd65fa20908150948k403c52d0t99b80ae20656546b@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>
	<5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com>
	<acd65fa20908150948k403c52d0t99b80ae20656546b@mail.gmail.com>
Message-ID: <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com>

On Sat, Aug 15, 2009 at 5:48 PM, Alexandre
Vassalotti<alexandre at peadrop.com> wrote:
> On Sat, Aug 15, 2009 at 12:25 PM, Mark Dickinson<dickinsm at gmail.com> wrote:
>> You say that the 'primary purpose of the API is long serialization'.
>> I'd argue that that's not quite true.
>
> And you are totally right. [...]

Eh?  But I came here for an argument!  Isn't this room 12?

A couple of other things:

If these additions to int go in, then presumably the _PyLong_AsBytes
and _PyLong_FromBytes functions should be documented and made public
(and have their leading underscores removed, too).  Those functions have
been stable for a good while, and are well-used within the Python source;
I think they're robust enough for public consumption.  There may be
some additional argument validation required;  I'll take a look at this.

In the issue tracker, Josiah Carlson asked about the possibility of
backporting to 2.7.  I can't see any problem with this, though there
would be some small extra work involved in making things work
for int as well as long.  Does anyone else see any issues with this?

Mark


From greg.ewing at canterbury.ac.nz  Sun Aug 16 03:06:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Aug 2009 13:06:35 +1200
Subject: [Python-ideas] Interrogate alternate namespace keyword
	and	concept
In-Reply-To: <90df837e0908141659k32dffb8fod69bf647736193f4@mail.gmail.com>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com>
	<d8cc6f9d0908141315w6cbe0b8di36986a1eaf7e77e9@mail.gmail.com>
	<90df837e0908141659k32dffb8fod69bf647736193f4@mail.gmail.com>
Message-ID: <4A875B9B.7000001@canterbury.ac.nz>

Prozacgod wrote:

> After this discussion I'm wondering why it ever ended up in any language 
> to begin with?

Pascal has it because it doesn't have any general way of
binding a local name to some nested part of a data structure.

You can always do this in Python, and also in C using the
& operator, so those languages don't need a Pascal-style
with-statement.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Sun Aug 16 03:30:05 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Aug 2009 13:30:05 +1200
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com>
	<200908081554.36358.steve@pearwood.info>
	<ca471dc20908081431u649096f8x729b0a41fecfc9ea@mail.gmail.com>
	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>
Message-ID: <4A87611D.7080903@canterbury.ac.nz>

Alexandre Vassalotti wrote:

> Whether it should default to 'little' or 'big' is pretty much an
> arbitrary choice. In my patch, I choose to default big-endian since it
> is the standard network byte-order. But maybe the option should
> default to little-endian instead since it more widely used.

This all sounds to me like an argument for not having
a default at all.

-- 
Greg


From ncoghlan at gmail.com  Sun Aug 16 03:23:39 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Aug 2009 11:23:39 +1000
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	<5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com>	<4A7FF6CE.7070903@gmail.com>	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>	<4A801D0C.2000502@gmail.com>	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>	<5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com>	<acd65fa20908150948k403c52d0t99b80ae20656546b@mail.gmail.com>
	<5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com>
Message-ID: <4A875F9B.5070701@gmail.com>

Mark Dickinson wrote:
> In the issue tracker, Josiah Carlson asked about the possibility of
> backporting to 2.7.  I can't see any problem with this, though there
> would be some small extra work involved in making things work
> for int as well as long.  Does anyone else see any issues with this?

What would we be converting them to in that case? 2.x strings? (I don't
have a problem with that, just pointing out there may be some additional
work due to changing the target type).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From greg.ewing at canterbury.ac.nz  Sun Aug 16 03:09:59 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Aug 2009 13:09:59 +1200
Subject: [Python-ideas] Interrogate alternate namespace keyword
	and	concept
In-Reply-To: <200908151213.23078.steve@pearwood.info>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>
	<90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com>
	<200908151213.23078.steve@pearwood.info>
Message-ID: <4A875C67.2080501@canterbury.ac.nz>

Steven D'Aprano wrote:
> if you want to know 
> how long it takes to make fresh stir-fried chicken, you have to include 
> the ten minutes it takes to chop up the vegetables and chicken,

Don't forget to include the time required to catch and kill
the chicken as well!

-- 
Greg


From steve at pearwood.info  Sun Aug 16 08:17:52 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 16 Aug 2009 16:17:52 +1000
Subject: [Python-ideas]
	=?iso-8859-1?q?Interrogate_alternate_namespace_key?=
	=?iso-8859-1?q?word_and=09concept?=
In-Reply-To: <4A875C67.2080501@canterbury.ac.nz>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<200908151213.23078.steve@pearwood.info>
	<4A875C67.2080501@canterbury.ac.nz>
Message-ID: <200908161617.53682.steve@pearwood.info>

On Sun, 16 Aug 2009 11:09:59 am Greg Ewing wrote:
> Steven D'Aprano wrote:
> > if you want to know
> > how long it takes to make fresh stir-fried chicken, you have to
> > include the ten minutes it takes to chop up the vegetables and
> > chicken,
>
> Don't forget to include the time required to catch and kill
> the chicken as well!

"In order to make an apple pie from scratch, you must first create the 
universe." -- Carl Sagan

Of course all timing tests make certain assumptions, and the art of 
optimization is partly to recognise when those assumptions are 
violated. If you do have a situation where you are executing the exact 
same code repeatedly, then pre-compiling is a good optimization over 
calling exec on the string directly. But you can't exclude the time for 
pre-compiling in the general case.


-- 
Steven D'Aprano


From alexandre at peadrop.com  Sun Aug 16 20:44:30 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Sun, 16 Aug 2009 14:44:30 -0400
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> 
	<4A7FF6CE.7070903@gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> 
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> 
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> 
	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com> 
	<5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> 
	<acd65fa20908150948k403c52d0t99b80ae20656546b@mail.gmail.com> 
	<5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com>
Message-ID: <acd65fa20908161144m14a04044pd8b9100880cdc0c1@mail.gmail.com>

On Sat, Aug 15, 2009 at 2:42 PM, Mark Dickinson<dickinsm at gmail.com> wrote:
> On Sat, Aug 15, 2009 at 5:48 PM, Alexandre
> Vassalotti<alexandre at peadrop.com> wrote:
>> On Sat, Aug 15, 2009 at 12:25 PM, Mark Dickinson<dickinsm at gmail.com> wrote:
>>> You say that the 'primary purpose of the API is long serialization'.
>>> I'd argue that that's not quite true.
>>
>> And you are totally right. [...]
>
> Eh? ?But I came here for an argument! ?Isn't this room 12?
>

Oh, oh, I am sorry. This is agreement. You want 12A, next door. =)

> A couple of other things:
>
> If these additions to int go in, then presumably the _PyLong_AsBytes
> and _PyLong_FromBytes functions should be documented and made public
> (and have their leading underscores removed, too).

You are referring to _PyLong_FromByteArray and _PyLong_AsByteArray, right?

-- Alexandre


From tleeuwenburg at gmail.com  Mon Aug 17 05:05:15 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Mon, 17 Aug 2009 13:05:15 +1000
Subject: [Python-ideas] Python 3.1 'api tracker' idea
Message-ID: <43c8685c0908162005h5d028d0r114f6a82b95c1e1@mail.gmail.com>

I use Python at work, but I depend on Numeric, Scientific and (and some
others).

It would be neat if there were a website I could use to check whether all my
external libraries were supported under Python 3.x. If I could tick boxes to
quickly check whether Python 3 could meet my needs -- or better yet register
with an email daemon which would email me when Python 3.x became useful for
me -- that would be awesome.

Cheers,
-T

-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090817/3f3d3cd0/attachment.html>

From stefan_ml at behnel.de  Mon Aug 17 15:04:02 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 17 Aug 2009 15:04:02 +0200
Subject: [Python-ideas] Python 3.1 'api tracker' idea
In-Reply-To: <43c8685c0908162005h5d028d0r114f6a82b95c1e1@mail.gmail.com>
References: <43c8685c0908162005h5d028d0r114f6a82b95c1e1@mail.gmail.com>
Message-ID: <h6bkg2$ga2$1@ger.gmane.org>

Tennessee Leeuwenburg wrote:
> I use Python at work, but I depend on Numeric, Scientific and (and some
> others).
> 
> It would be neat if there were a website I could use to check whether all my
> external libraries were supported under Python 3.x. If I could tick boxes to
> quickly check whether Python 3 could meet my needs -- or better yet register
> with an email daemon which would email me when Python 3.x became useful for
> me -- that would be awesome.

Have you searched PyPI for Python 3 compliant packages? Everything that
doesn't list itself there as supporting Py3 likely doesn't support it.

Stefan



From mcaninch at lanl.gov  Thu Aug 20 01:07:54 2009
From: mcaninch at lanl.gov (Jeff McAninch)
Date: Wed, 19 Aug 2009 17:07:54 -0600
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
Message-ID: <4A8C85CA.6010306@lanl.gov>

I would like to propose an expression, similar to the if-else expression,
that responds to exceptions.

I had originally posted this (probably mistakenly) on py-dev.  This 
current posting is a cleaned up
version of the idea, based on responses I got on from the earlier posting.

_*Abstract:
*_Proposal for a conditional expression, similar to the if-else 
expression, that responds to exceptions.

_*Motivation:
*_An expression syntax that responds to exceptions, and which reproduces 
the readability and conciseness of the if-else conditional expression, 
would simplify some exception-handling cases, especially within list 
comprehensions.

_*Very Simple Example - type coercion:
*_Current approach:
    try:
        x = float(string)
    except:
        x = float('nan')

Proposed solution using exception-based conditional expression:
    x = float(string) except ValueError: float('nan')


_*Simple Example - type coercion in a list comprehension:
*_Current approach:
    def safe_float(string):
        try:
            x = float(string)
        except ValueError:
            x = float('nan')
        return x
    ...
    xs = (safe(float(string)) for string in strings)

Proposed solution using exception-based conditional expression:
    xs = ((float(string) except ValueError: float('nan')) for string in 
strings)

_*Discussion:
*_In my own python coding, I find I make common use of the if-else 
conditional expression, especially within list comprehensions.  (In one 
of my packages, which has ~5800 lines of code, I found if-else 
expressions in ~1% of the lines.)

Here is a slightly more involved example than the examples presented 
above.  In data processing, I often string together a sequence of 
iterable list comprehensions, corresponding to a sequence of operations 
on a given dataset "ys" to produce a processed dataset "x":
    xs = (operation_A(x) for x in ys)
    xs = (operation_B(x) for x in xs if filter_B(x))
    xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for 
x in xs if filter_C(x))
    # final, explicit list of values
    xs = [ x for x in xs ]
This is often a flexible way for me to define processing and filtering 
sequences which also seems
to have good performance on very large datasets.  One advantage is that 
I can quickly mix-and-match from existing processes like this to make a 
new process.  An exception-based conditional would go nicely
into many of these process sequences, keeping them both robust and flexible.
    xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)

I also often have object classes which have some common method or 
attribute.  For instance, some of my objects have scope-dependent values:
    x = y.evaluate(scope))
where scope is typically locals(), globals(), or some other 
dictionary-like container.  But, to keep my code modular, I want to 
handle, in the same lines of code, objects which do not have some 
particular method, which leads me to lines of code like:
    x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y
This seems not very "Pythonic", similar to using type-testing instead of 
try-except.  (My impression was that there was a long-standing trend in 
the evolution of Python to remove tests like this, and I thought that 
was the original motivation for the try-except syntax.)

I would much rather write:
    x = y.evaluate(locals()) except AttributeError: y
or, in the list comprehension example:
    xs = (y.evaluate(locals()) except AttributeError: y for y in ys)

Clearly this can be handled in several ways with the language as it is.  
One way is to define a new function, as in the second simple example above:
    def safe_evaluate(y,scope):
       try:
          x = y.evaluate(scope)
       except AttributeError:
          x = y
       return x
    ...
    xs = (safe_evaluate(y,locals()) for y in ys)
but this quickly (in my packages at least) leads to an annoying 
proliferation of "safe_" functions.
Again, this seems not to be in the "Pythonic" spirit, and is also less 
concise, less readable.  (I also suspect, but have not verified, that 
this is in general less efficient than in-line expressions -- wasn't 
that part of the original motivation for list comprehensions?).

In the thread of my previous post to py-dev, there were comments, 
questions, and suggestions concerning the details of the syntax.  Having 
reflected on this for a couple weeks, I am now most strongly supportive 
of what is essentially just an inline compression of the current 
try-except syntax.  So the following examples would be allowed:
    x = expression0 except: default_expression
    x = expression0 except exception1: expression1 except exception2: 
expression2 except: default_expression

Or, more generally:
    x = expression0\
            except exception1: expression1\
            except exception2: expression2\
            ...
            except exceptionI: expressionI\
            ...
            except: default_expression
In this last example, the behaviour would be as follows:
    - evaluate expression0. 
            If no exception is encountered, return the result.
    - if an exception is encountered,
            search for the matching exception in the except clauses.
    - if a matching exception ("exceptionI") is found,
            evaluate the corresponding expression ("expressionI"), and 
return the result.
    - if no matching exception is found, and a default except: clause 
(i.e., one without and exception)
          is given, evaluate default_expression, and return the result.
    - if no matching exception is found, and no default except clause if 
given,
          pass the exception on to the caller.
    - if a new exception is encountered while evaluating an an except 
expression ("expressionI"),
          pass the exception on to the caller.

I hope I have made a convincing case here.  This seems to me to be a 
natural ("Pythonic") addition to the language.

Jeff McAninch

-- 
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch at lanl.gov
==========================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090819/9e8a0738/attachment.html>

From tleeuwenburg at gmail.com  Thu Aug 20 01:59:31 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 20 Aug 2009 09:59:31 +1000
Subject: [Python-ideas] exception based conditional expression,
	similar to 	if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <43c8685c0908191659h73567060h2075cd4390b19eb0@mail.gmail.com>

I like it. I'm not fully competent to comment on the details, but a big +1
for something like this, if not this.

-T

On Thu, Aug 20, 2009 at 9:07 AM, Jeff McAninch <mcaninch at lanl.gov> wrote:

>  I would like to propose an expression, similar to the if-else expression,
> that responds to exceptions.
>
> I had originally posted this (probably mistakenly) on py-dev.  This current
> posting is a cleaned up
> version of the idea, based on responses I got on from the earlier posting.
>
> *Abstract:
> *Proposal for a conditional expression, similar to the if-else expression,
> that responds to exceptions.
>
> *Motivation:
> *An expression syntax that responds to exceptions, and which reproduces
> the readability and conciseness of the if-else conditional expression, would
> simplify some exception-handling cases, especially within list
> comprehensions.
>
> *Very Simple Example - type coercion:
> *Current approach:
>     try:
>         x = float(string)
>     except:
>         x = float('nan')
>
> Proposed solution using exception-based conditional expression:
>     x = float(string) except ValueError: float('nan')
>
>
> *Simple Example - type coercion in a list comprehension:
> *Current approach:
>     def safe_float(string):
>         try:
>             x = float(string)
>         except ValueError:
>             x = float('nan')
>         return x
>     ...
>     xs = (safe(float(string)) for string in strings)
>
> Proposed solution using exception-based conditional expression:
>     xs = ((float(string) except ValueError: float('nan')) for string in
> strings)
>
> *Discussion:
> *In my own python coding, I find I make common use of the if-else
> conditional expression, especially within list comprehensions.  (In one of
> my packages, which has ~5800 lines of code, I found if-else expressions in
> ~1% of the lines.)
>
> Here is a slightly more involved example than the examples presented
> above.  In data processing, I often string together a sequence of iterable
> list comprehensions, corresponding to a sequence of operations on a given
> dataset "ys" to produce a processed dataset "x":
>     xs = (operation_A(x) for x in ys)
>     xs = (operation_B(x) for x in xs if filter_B(x))
>     xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x
> in xs if filter_C(x))
>     # final, explicit list of values
>     xs = [ x for x in xs ]
> This is often a flexible way for me to define processing and filtering
> sequences which also seems
> to have good performance on very large datasets.  One advantage is that I
> can quickly mix-and-match from existing processes like this to make a new
> process.  An exception-based conditional would go nicely
> into many of these process sequences, keeping them both robust and
> flexible.
>     xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in
> xs)
>
> I also often have object classes which have some common method or
> attribute.  For instance, some of my objects have scope-dependent values:
>     x = y.evaluate(scope))
> where scope is typically locals(), globals(), or some other dictionary-like
> container.  But, to keep my code modular, I want to handle, in the same
> lines of code, objects which do not have some particular method, which leads
> me to lines of code like:
>     x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y
> This seems not very "Pythonic", similar to using type-testing instead of
> try-except.  (My impression was that there was a long-standing trend in the
> evolution of Python to remove tests like this, and I thought that was the
> original motivation for the try-except syntax.)
>
> I would much rather write:
>     x = y.evaluate(locals()) except AttributeError: y
> or, in the list comprehension example:
>     xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
>
> Clearly this can be handled in several ways with the language as it is.
> One way is to define a new function, as in the second simple example above:
>     def safe_evaluate(y,scope):
>        try:
>           x = y.evaluate(scope)
>        except AttributeError:
>           x = y
>        return x
>     ...
>     xs = (safe_evaluate(y,locals()) for y in ys)
> but this quickly (in my packages at least) leads to an annoying
> proliferation of "safe_" functions.
> Again, this seems not to be in the "Pythonic" spirit, and is also less
> concise, less readable.  (I also suspect, but have not verified, that this
> is in general less efficient than in-line expressions -- wasn't that part of
> the original motivation for list comprehensions?).
>
> In the thread of my previous post to py-dev, there were comments,
> questions, and suggestions concerning the details of the syntax.  Having
> reflected on this for a couple weeks, I am now most strongly supportive of
> what is essentially just an inline compression of the current try-except
> syntax.  So the following examples would be allowed:
>     x = expression0 except: default_expression
>     x = expression0 except exception1: expression1 except exception2:
> expression2 except: default_expression
>
> Or, more generally:
>     x = expression0\
>             except exception1: expression1\
>             except exception2: expression2\
>             ...
>             except exceptionI: expressionI\
>             ...
>             except: default_expression
> In this last example, the behaviour would be as follows:
>     - evaluate expression0.
>             If no exception is encountered, return the result.
>     - if an exception is encountered,
>             search for the matching exception in the except clauses.
>     - if a matching exception ("exceptionI") is found,
>             evaluate the corresponding expression ("expressionI"), and
> return the result.
>     - if no matching exception is found, and a default except: clause
> (i.e., one without and exception)
>           is given, evaluate default_expression, and return the result.
>     - if no matching exception is found, and no default except clause if
> given,
>           pass the exception on to the caller.
>     - if a new exception is encountered while evaluating an an except
> expression ("expressionI"),
>           pass the exception on to the caller.
>
> I hope I have made a convincing case here.  This seems to me to be a
> natural ("Pythonic") addition to the language.
>
> Jeff McAninch
>
> --
> ==========================
> Jeffrey E. McAninch, PhD
> Physicist, X-2-IFD
> Los Alamos National Laboratory
> Phone: 505-667-0374
> Email: mcaninch at lanl.gov
> ==========================
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090820/89898f83/attachment.html>

From ironfroggy at gmail.com  Thu Aug 20 03:59:07 2009
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Wed, 19 Aug 2009 21:59:07 -0400
Subject: [Python-ideas] exception based conditional expression,
	similar to 	if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <76fd5acf0908191859t44f77d93ke909d8059b939d69@mail.gmail.com>

-1 on colons in the expression like that. I like the idea of being
able to handle an exception in generator expressions and the like, but
I've never seen a syntax I liked. I think I've favored the idea of
something like `float(x) except float('nan') if ValueError` thinking
it reads more naturally as an expression, puts the real logic
("convert x to a float or get a NaN float") together, which I think
makes sense.

I fear being able to "express" too much. That is, we're going to
remove all that indentation gives us if eventually everything is given
an expression form.

Let's add an import expression, next. Maybe this?

sys.exit(1) from sys

Joke, obviously.

On Wed, Aug 19, 2009 at 7:07 PM, Jeff McAninch<mcaninch at lanl.gov> wrote:
> I would like to propose an expression, similar to the if-else expression,
> that responds to exceptions.
>
> I had originally posted this (probably mistakenly) on py-dev.? This current
> posting is a cleaned up
> version of the idea, based on responses I got on from the earlier posting.
>
> Abstract:
> Proposal for a conditional expression, similar to the if-else expression,
> that responds to exceptions.
>
> Motivation:
> An expression syntax that responds to exceptions, and which reproduces the
> readability and conciseness of the if-else conditional expression, would
> simplify some exception-handling cases, especially within list
> comprehensions.
>
> Very Simple Example - type coercion:
> Current approach:
> ??? try:
> ??????? x = float(string)
> ??? except:
> ??????? x = float('nan')
>
> Proposed solution using exception-based conditional expression:
> ??? x = float(string) except ValueError: float('nan')
>
>
> Simple Example - type coercion in a list comprehension:
> Current approach:
> ??? def safe_float(string):
> ??????? try:
> ??????????? x = float(string)
> ??????? except ValueError:
> ??????????? x = float('nan')
> ??????? return x
> ??? ...
> ??? xs = (safe(float(string)) for string in strings)
>
> Proposed solution using exception-based conditional expression:
> ??? xs = ((float(string) except ValueError: float('nan')) for string in
> strings)
>
> Discussion:
> In my own python coding, I find I make common use of the if-else conditional
> expression, especially within list comprehensions.? (In one of my packages,
> which has ~5800 lines of code, I found if-else expressions in ~1% of the
> lines.)
>
> Here is a slightly more involved example than the examples presented above.
> In data processing, I often string together a sequence of iterable list
> comprehensions, corresponding to a sequence of operations on a given dataset
> "ys" to produce a processed dataset "x":
> ??? xs = (operation_A(x) for x in ys)
> ??? xs = (operation_B(x) for x in xs if filter_B(x))
> ??? xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in
> xs if filter_C(x))
> ??? # final, explicit list of values
> ??? xs = [ x for x in xs ]
> This is often a flexible way for me to define processing and filtering
> sequences which also seems
> to have good performance on very large datasets.? One advantage is that I
> can quickly mix-and-match from existing processes like this to make a new
> process.? An exception-based conditional would go nicely
> into many of these process sequences, keeping them both robust and flexible.
> ??? xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
>
> I also often have object classes which have some common method or
> attribute.? For instance, some of my objects have scope-dependent values:
> ??? x = y.evaluate(scope))
> where scope is typically locals(), globals(), or some other dictionary-like
> container.? But, to keep my code modular, I want to handle, in the same
> lines of code, objects which do not have some particular method, which leads
> me to lines of code like:
> ??? x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y
> This seems not very "Pythonic", similar to using type-testing instead of
> try-except.? (My impression was that there was a long-standing trend in the
> evolution of Python to remove tests like this, and I thought that was the
> original motivation for the try-except syntax.)
>
> I would much rather write:
> ??? x = y.evaluate(locals()) except AttributeError: y
> or, in the list comprehension example:
> ??? xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
>
> Clearly this can be handled in several ways with the language as it is.? One
> way is to define a new function, as in the second simple example above:
> ??? def safe_evaluate(y,scope):
> ??? ?? try:
> ??? ?? ?? x = y.evaluate(scope)
> ??? ?? except AttributeError:
> ??? ?? ?? x = y
> ??? ?? return x
> ??? ...
> ??? xs = (safe_evaluate(y,locals()) for y in ys)
> but this quickly (in my packages at least) leads to an annoying
> proliferation of "safe_" functions.
> Again, this seems not to be in the "Pythonic" spirit, and is also less
> concise, less readable.? (I also suspect, but have not verified, that this
> is in general less efficient than in-line expressions -- wasn't that part of
> the original motivation for list comprehensions?).
>
> In the thread of my previous post to py-dev, there were comments, questions,
> and suggestions concerning the details of the syntax.? Having reflected on
> this for a couple weeks, I am now most strongly supportive of what is
> essentially just an inline compression of the current try-except syntax.? So
> the following examples would be allowed:
> ??? x = expression0 except: default_expression
> ??? x = expression0 except exception1: expression1 except exception2:
> expression2 except: default_expression
>
> Or, more generally:
> ??? x = expression0\
> ? ? ? ? ? ? except exception1: expression1\
> ??????????? except exception2: expression2\
> ??????????? ...
> ??????????? except exceptionI: expressionI\
> ??????????? ...
> ??????????? except: default_expression
> In this last example, the behaviour would be as follows:
> ??? - evaluate expression0.
> ??????????? If no exception is encountered, return the result.
> ??? - if an exception is encountered,
> ??????????? search for the matching exception in the except clauses.
> ??? - if a matching exception ("exceptionI") is found,
> ??????????? evaluate the corresponding expression ("expressionI"), and
> return the result.
> ??? - if no matching exception is found, and a default except: clause (i.e.,
> one without and exception)
> ? ? ? ??? is given, evaluate default_expression, and return the result.
> ??? - if no matching exception is found, and no default except clause if
> given,
> ????????? pass the exception on to the caller.
> ??? - if a new exception is encountered while evaluating an an except
> expression ("expressionI"),
> ????????? pass the exception on to the caller.
>
> I hope I have made a convincing case here.? This seems to me to be a natural
> ("Pythonic") addition to the language.
>
> Jeff McAninch
>
> --
> ==========================
> Jeffrey E. McAninch, PhD
> Physicist, X-2-IFD
> Los Alamos National Laboratory
> Phone: 505-667-0374
> Email: mcaninch at lanl.gov
> ==========================
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy


From stephen at xemacs.org  Thu Aug 20 04:31:20 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 20 Aug 2009 11:31:20 +0900
Subject: [Python-ideas]  exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <87eir75i87.fsf@uwakimon.sk.tsukuba.ac.jp>

Jeff McAninch writes:

 > _*Very Simple Example - type coercion:
 > *_Current approach:
 >     try:
 >         x = float(string)
 >     except:
 >         x = float('nan')
 > 
 > Proposed solution using exception-based conditional expression:
 >     x = float(string) except ValueError: float('nan')

-1

I really don't like the colon in the middle of the expression.  It
just looks like broken syntax because everywhere else in Python colon
introduces a suite.  I wonder if you really need it?

I generally don't like a general conditional expression.  For reasons
presented below, I don't think it's as flexible as think it can be.
OTOH, I fund it much less readable than the existing statement-based
construction.

 > _*Simple Example - type coercion in a list comprehension:
 > *_Current approach:
 >     def safe_float(string):
 >         try:
 >             x = float(string)
 >         except ValueError:
 >             x = float('nan')
 >         return x
 >     ...
 >     xs = (safe(float(string)) for string in strings)

I don't have a problem with the above.  But I can see that for

 > In data processing, I often string together a sequence of iterable
 > list comprehensions, corresponding to a sequence of operations on a
 > given dataset "ys" to produce a processed dataset "x":

an exception-handling clause in comprehensions (and generator
expressions?) might be useful, to make the exceptions handled
explicit.  For the cases I can imagine using myself, I would generally
prefer the approach of defining functions to handle the exceptions,
because they'd all be similar (eg, coercing exceptions to float("nan")
as above), though.

 > try-except syntax.  So the following examples would be allowed:
 >     x = expression0 except: default_expression
 >     x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression

But it seems to me the ez-read interpretation of

    x = expression0 except exception1: expression1 except: default_expression

is

    x = expression0 except exception1: (expression1 except: default_expression)

Ie, your parser resolves the "dangling except" ambiguity in the
opposite way to the conventional resolution of the "dangling else"
ambiguity.  And what if excepts are mixed with conditional
expressions?

  x = exp0 except exc1: exp1 if exp2 else exp3 except: exp4

Does the bare except bind to exp0, the if expression, or exp3?  I'm
sure you can define rules to disambiguate.  However, I suspect that it
will be hard to get agreement that any given set of rules is the
appropriate way to resolve the ambiguity.


From dstanek at dstanek.com  Thu Aug 20 04:28:35 2009
From: dstanek at dstanek.com (David Stanek)
Date: Wed, 19 Aug 2009 22:28:35 -0400
Subject: [Python-ideas] exception based conditional expression,
	similar to 	if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <de32cc030908191928s77c4bcdbn97dbf798d5c712fa@mail.gmail.com>

On Wed, Aug 19, 2009 at 7:07 PM, Jeff McAninch<mcaninch at lanl.gov> wrote:
> I would like to propose an expression, similar to the if-else expression,
> that responds to exceptions.

This is a pretty good idea. I can definitely see where I would use it
in my own code. The only thing I would caution is in your simple
example. I would choose to create a module full of conversion
functions. It violates DRY to have the 'try float otherwise Nan'
statement all over the place.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek


From mcaninch at lanl.gov  Thu Aug 20 12:15:28 2009
From: mcaninch at lanl.gov (Jeff McAninch)
Date: Thu, 20 Aug 2009 04:15:28 -0600
Subject: [Python-ideas] [Fwd: Re:  exception based conditional expression,
 similar to 	if-else conditional expression]
Message-ID: <4A8D2240.6080107@lanl.gov>

Calvin Spealman wrote:

> -1 on colons in the expression like that. I like the idea of being
> able to handle an exception in generator expressions and the like, but
> I've never seen a syntax I liked. I think I've favored the idea of
> something like `float(x) except float('nan') if ValueError` thinking
> it reads more naturally as an expression, puts the real logic
> ("convert x to a float or get a NaN float") together, which I think
> makes sense.
>
>   

Yes, I agree about the colons.  They have no purpose.  I was just 
blindly following the try-except.  (Duh! on my part)

So, in the simple example:
   x = float(string) except ValueError float('nan')

But possibly the exception tuples now have to be explicitly tuples?
   x = float(string) except (ValueError,) float('nan')

So the general case would be something like
   exception_expression :== nominal_value {except exception_tuple exception_value}* {except default_value}


Jeff McAninch


-- 
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch at lanl.gov
==========================



From mcaninch at lanl.gov  Thu Aug 20 12:30:02 2009
From: mcaninch at lanl.gov (Jeff McAninch)
Date: Thu, 20 Aug 2009 04:30:02 -0600
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <87eir75i87.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4A8C85CA.6010306@lanl.gov>
	<87eir75i87.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4A8D25AA.4080502@lanl.gov>

Stephen J. Turnbull wrote:
> Ie, your parser resolves the "dangling except" ambiguity in the
> opposite way to the conventional resolution of the "dangling else"
> ambiguity.  And what if excepts are mixed with conditional
> expressions?
>
>   x = exp0 except exc1: exp1 if exp2 else exp3 except: exp4
>
> Does the bare except bind to exp0, the if expression, or exp3?  I'm
> sure you can define rules to disambiguate.  However, I suspect that it
> will be hard to get agreement that any given set of rules is the
> appropriate way to resolve the ambiguity.
>   

Following up on my previous post, I think the general syntax would be 
something like:
    exception_expression :== nominal_value {except exception_tuple 
exception_value}* {except default_value}

Hopefully this disambiguates the issue, at least from the parser point 
of view.  Requiring an explicit tuple of exceptions (ie., requiring the 
parens around the tuple of exceptions) makes sense too, and I thought I 
had seen other discussions of requiring explicit tuples where currently 
parens could be implied.

With the definition above, I think the combinations of exception 
expressions with if-else conditional expressions is probably also 
unambiguous (though not necessarily easily read).  I haven't sat down to 
verify this mathematically yet though.

Allowing parens around the except clauses might help to make it more 
readable?
    x = float(string) (except (ValueError,) float('nan')) (except 
(SomeOtherException,) someOtherValue)
Okay, maybe not?

(BTW: thanks for the cool new verb -- disambiguate -- though my friends 
and family may not be so happy when I start using it at every opportunity!)

-- 
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch at lanl.gov
==========================



From pyideas at rebertia.com  Thu Aug 20 12:45:31 2009
From: pyideas at rebertia.com (Chris Rebert)
Date: Thu, 20 Aug 2009 03:45:31 -0700
Subject: [Python-ideas] [Fwd: Re: exception based conditional expression,
	similar to if-else conditional expression]
In-Reply-To: <4A8D2240.6080107@lanl.gov>
References: <4A8D2240.6080107@lanl.gov>
Message-ID: <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com>

On Thu, Aug 20, 2009 at 3:15 AM, Jeff McAninch<mcaninch at lanl.gov> wrote:
> Calvin Spealman wrote:
>
>> -1 on colons in the expression like that. I like the idea of being
>> able to handle an exception in generator expressions and the like, but
>> I've never seen a syntax I liked. I think I've favored the idea of
>> something like `float(x) except float('nan') if ValueError` thinking
>> it reads more naturally as an expression, puts the real logic
>> ("convert x to a float or get a NaN float") together, which I think
>> makes sense.
>>
>>
>
> Yes, I agree about the colons. ?They have no purpose. ?I was just blindly
> following the try-except. ?(Duh! on my part)
>
> So, in the simple example:
> ?x = float(string) except ValueError float('nan')

I would feel more comfortable with another keyword in there. Perhaps:

x = float(string) except ValueError then float('nan')

Which reads like:

"x is the float value of string, except in the event of ValueError,
then it's float('nan')"

Which I find rather pleasing.

> But possibly the exception tuples now have to be explicitly tuples?
> ?x = float(string) except (ValueError,) float('nan')

Yech. That doesn't look so great.


At any rate, I do seriously question the bang-for-the-buck of adding
this construct. Yes, it makes such conversions shorter, but couldn't
one just define a `float_or_nan` conversion function that defaults to
NaN in case of error to essentially the same effect?
I can see how the construct might help in comparatively
quick-and-dirty scripts, but should Python actively encourage
comparatively cavalier error-handling?

Cheers,
Chris
--
http://blog.rebertia.com


From masklinn at masklinn.net  Thu Aug 20 13:17:08 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 20 Aug 2009 13:17:08 +0200
Subject: [Python-ideas] exception based conditional expression,
	similar to if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <93EF2F8E-3BD1-4243-83DE-5FC0E6FD58D2@masklinn.net>

On 20 Aug 2009, at 01:07 , Jeff McAninch wrote:
> I would like to propose an expression, similar to the if-else  
> expression,
> that responds to exceptions.
-1, if this goes through it will be extended to all other compound  
statements over time and we'll end up with 2 or 3 ways to do every  
thing.

A much better idea would be to find a way to make all compound  
statements into expressions, future-proofing the decision and avoiding  
the redundancy between "compound statement statement" and "compound  
statement expression".


From mcaninch at lanl.gov  Thu Aug 20 13:18:42 2009
From: mcaninch at lanl.gov (Jeff McAninch)
Date: Thu, 20 Aug 2009 05:18:42 -0600
Subject: [Python-ideas] [Fwd: Re: exception based conditional expression,
 similar to if-else conditional expression]
In-Reply-To: <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com>
References: <4A8D2240.6080107@lanl.gov>
	<50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com>
Message-ID: <4A8D3112.7030905@lanl.gov>

Chris Rebert wrote:
> Yech. That doesn't look so great.
>
I agree about the parens, but thought I should put it out there.  The 
parens around the exception clauses should not be required, but should 
be allowed to reduce ambiguity,  In my own case, I would tend to use 
continuation lines to make multi-exception expressions more readable.

> At any rate, I do seriously question the bang-for-the-buck of adding
> this construct. Yes, it makes such conversions shorter, but couldn't
> one just define a `float_or_nan` conversion function that defaults to
> NaN in case of error to essentially the same effect?
> I can see how the construct might help in comparatively
> quick-and-dirty scripts, but should Python actively encourage
> comparatively cavalier error-handling?
>
>   
One of the surprising things about Python is the impressive performance 
that can be achieved, even though it is an interpreted langauge.  In my 
experience, this is one of the aspects of Python that sets it apart from 
other interpreted environments, and why I can apply Python to heavy duty 
scientific computing in the same environment where I pop up a gui or 
parse a text file.

This performance is achieved as one increases the ratio of work done in 
compiled code, to the amount of work being done by the interpreter.

Hence, list comprehensions and iterator expressions, in place of for and 
while loops.

The bang-for-the-buck in the exception expression is in performance.  
This is not in any way just a quick-and-dirty issue.  It's exactly the 
same motivation as the if-else expression.

Please correct me if I am in error, but doesn't a call to a python 
function (as opposed to a compiled C-function) substantially slow down a 
list comprehension?  (I will endeavor to generate a test case this 
weekend and post the results, but this is consistent with everything 
I've read about optimizing Python code.) 

And remember, this is called "exception-handling" rather than 
"error-handling" for a reason.  In many practical cases, the exceptions 
can be a significant, if not dominant, fraction of the executions of the 
code.  That, I thought, was the logic behind try:except in the first 
place: treating exceptions as a natural part of the coding, rather than 
a collection of special cases.

I see this as encouraging more robust, rather than more cavalier, 
exception handling, and doing it in a more concise syntax (and I assert, 
more efficient from a performance perspective -- I'll try to put this 
assertion to a quantitative test this weekend.).

-- 
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch at lanl.gov
==========================



From p.f.moore at gmail.com  Thu Aug 20 16:02:28 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 20 Aug 2009 15:02:28 +0100
Subject: [Python-ideas] pep 312 - implicit lambdas idea
In-Reply-To: <4A7CCB94.8020408@canterbury.ac.nz>
References: <d8cc6f9d0908070546h3d5d950cy170b6f310b7bdff3@mail.gmail.com>
	<87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz>
Message-ID: <79990c6b0908200702l5aaf6b06kd799760c289c818c@mail.gmail.com>

2009/8/8 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> My suggestion for a lightweight lambda syntax is
>
> ?args -> expr
>
> Examples:
>
> ?map(x -> x + 5, something)
>
> ?d = defauldict(->[])

+1

Paul.


From jimjjewett at gmail.com  Thu Aug 20 16:25:06 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 20 Aug 2009 10:25:06 -0400
Subject: [Python-ideas] [Fwd: Re: exception based conditional expression,
	similar to if-else conditional expression]
In-Reply-To: <4A8D3112.7030905@lanl.gov>
References: <4A8D2240.6080107@lanl.gov>
	<50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> 
	<4A8D3112.7030905@lanl.gov>
Message-ID: <fb6fbf560908200725h7ffc8aeco240aad33390671d5@mail.gmail.com>

On Thu, Aug 20, 2009 at 7:18 AM, Jeff McAninch<mcaninch at lanl.gov> wrote:
> The bang-for-the-buck in the exception expression
> is in performance. ...

> ... doesn't a call to a python function
> (as opposed to a compiled C-function) substantially
> slow down a list comprehension?

Probably, but you don't have to use a comprehension.

def g(seq):
    for e in seq:
        try:
            yield float(e)
        except:
            pass # whatever, even skipping the element

If you don't need it in generator form, then just collect the results
into a list and return that instead of yielding.  (And obviously, that
for loop doesn't even have to be in a separate (generator or) function
at all.)

-jJ


From steve at pearwood.info  Thu Aug 20 16:25:51 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 21 Aug 2009 00:25:51 +1000
Subject: [Python-ideas] exception based conditional expression,
	similar to if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <200908210025.51640.steve@pearwood.info>

On Thu, 20 Aug 2009 09:07:54 am Jeff McAninch wrote:
> I would like to propose an expression, similar to the if-else
> expression, that responds to exceptions.
...
> _*Simple Example - type coercion in a list comprehension:
> *_Current approach:
>     def safe_float(string):
>         try:
>             x = float(string)
>         except ValueError:
>             x = float('nan')
>         return x
>     ...
>     xs = (safe(float(string)) for string in strings)

Personally, I think the above is likely to be the approach in a serious 
application. For lightweight applications, an expression-based solution 
would be acceptable, but it has the disadvantage of not being able to 
be documented or tested independently.


> Proposed solution using exception-based conditional expression:
>     xs = ((float(string) except ValueError: float('nan')) for string
> in strings)

Others have suggested that the colon should be dropped. I dislike that 
idea, because there's nothing but whitespace delimiting the list of 
exceptions from the except-expression:

EXPR except EXCEPT-LIST EXCEPT-EXPR

As far as I can tell, that would be unprecedented in Python syntax.


...
> Here is a slightly more involved example than the examples presented
> above.  In data processing, I often string together a sequence of
> iterable list comprehensions, corresponding to a sequence of
> operations on a given dataset "ys" to produce a processed dataset
> "x":
>     xs = (operation_A(x) for x in ys)
>     xs = (operation_B(x) for x in xs if filter_B(x))
>     xs = (operation_C(x) if (some_condition(x)) else operation_D(x)
> for x in xs if filter_C(x))
>     # final, explicit list of values
>     xs = [ x for x in xs ]

Ouch! That last one is better written as:

xs = list(xs)

> This is often a flexible way for me to define processing and
> filtering sequences which also seems
> to have good performance on very large datasets.

Chaining iterators together is a powerful technique, and you don't have 
to be limited to generator expressions, you can use regular generators 
too:

def operate(xs):
    for x in xs:
         try:  yield operation_C(x)
         except ValueError: yield operation_D(x)

xs = (operation_A(x) for x in ys)
xs = (operation_B(x) for x in xs if filter_B(x))
xs = operate(xs)
# finally convert to a list
xs = list(xs)



> One advantage is 
> that I can quickly mix-and-match from existing processes like this to
> make a new process.  An exception-based conditional would go nicely
> into many of these process sequences, keeping them both robust and
> flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x)
> for x in xs)

So far this is an argument for iterator-based processing, not for 
exception-based conditionals :)

You can use exceptions in generator exceptions, you just have to put 
them into a function first. 


> I also often have object classes which have some common method or
> attribute.  For instance, some of my objects have scope-dependent
> values: x = y.evaluate(scope))
> where scope is typically locals(), globals(), or some other
> dictionary-like container.  But, to keep my code modular, I want to
> handle, in the same lines of code, objects which do not have some
> particular method, which leads me to lines of code like:
>     x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y

Is that real code? I don't think it works the way you think. Given a 
reasonable definition for y (an instance of a class with no special 
tricks) the if test will never succeed:


>>> class MyObject(object):
...     def evaluate(self, arg):
...             return arg
...
>>> y = MyObject()
>>> y.evaluate(42)
42
>>> 'evaluate' in y.__dict__
False


A better way of writing that would be:

x = y.evaluate(locals()) if hasattr(y, 'evaluate') else y

which have the bonus of working correctly even if y gets its evaluate() 
method via inheritance.

Another way, two lines instead of one:

try: x = y.evaluate(locals())
except AttributeError: x = y


A third method: make sure all objects you pass have an evaluate() 
method, even if they are a null-operation that just return self.



> This seems not very "Pythonic", similar to using type-testing instead
> of try-except.

You're talking about the Look Before You Leap technique rather than 
Easier To Ask Forgiveness Than Permission.

Both techniques are valid, and both have advantages. ETAFTP is 
advantageous when exceptions are rare and hard to predict; LBYL is 
better if exceptions are common and easy to predict.


> In the thread of my previous post to py-dev, there were comments,
> questions, and suggestions concerning the details of the syntax. 
> Having reflected on this for a couple weeks, I am now most strongly
> supportive of what is essentially just an inline compression of the
> current try-except syntax.  So the following examples would be
> allowed: 
> x = expression0 except: default_expression 

I would argue strongly that allowing a bare except is a SERIOUS mistake. 
Bare excepts have very few legitimate uses, and now that string 
exceptions are deprecated and very rare, it's no hardship to write:

except Exception:

if you want to catch everything except KeyboardInterrupt and one or two 
other things, or:

except BaseException:

for those rare times you actually do want to catch everything.



-- 
Steven D'Aprano


From mcaninch at lanl.gov  Thu Aug 20 16:27:26 2009
From: mcaninch at lanl.gov (Jeff McAninch)
Date: Thu, 20 Aug 2009 08:27:26 -0600
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
Message-ID: <4A8D5D4E.5090108@lanl.gov>

>
> -1, if this goes through it will be extended to all other compound  
> statements over time and we'll end up with 2 or 3 ways to do every  
> thing.
>
> A much better idea would be to find a way to make all compound  
> statements into expressions, future-proofing the decision and avoiding  
> the redundancy between "compound statement statement" and "compound  
> statement expression".
>   
Python's list of compound statements is pretty thin (just 7),
and 4 of these already have expression forms.

    for     ==> list comprehension, iterator expression
    while   ==> list comprehension, iterator
    if      ==> if-else conditional expression
    def     ==> already have lambda expressions

    try     ==> my proposal

    class   ==> Okay, so this one doesn't have a compound statement,
                   but where would you use it?
    with    ==> I haven't had an occasion to use with,
                   so can't think of an example where it could
                   be useful as a compound expression.

Seems to me try-except is kind of sticking out here, and I've
tried to show what I think are natural and common places where
a try-except expression would be appropriate.

-- 
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch at lanl.gov
==========================



From steve at pearwood.info  Thu Aug 20 16:46:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 21 Aug 2009 00:46:38 +1000
Subject: [Python-ideas] exception based conditional expression,
	similar to if-else conditional expression
In-Reply-To: <4A8D5D4E.5090108@lanl.gov>
References: <4A8D5D4E.5090108@lanl.gov>
Message-ID: <200908210046.39235.steve@pearwood.info>

On Fri, 21 Aug 2009 12:27:26 am Jeff McAninch wrote:

> Python's list of compound statements is pretty thin (just 7),
> and 4 of these already have expression forms.
>
>     for     ==> list comprehension, iterator expression
>     while   ==> list comprehension, iterator
>     if      ==> if-else conditional expression

But that one is ugly (that is, I don't think anyone *likes* the syntax, 
it's just considered the least worst), and some of us consider that its 
usefulness is dubious, or at least weak. I think the main reason that 
Python has a ternary if statement is to stop people writing the 
error-prone:

cond and x or y

as a short-circuit ternary if.



>     def     ==> already have lambda expressions
>
>     try     ==> my proposal
>
>     class   ==> Okay, so this one doesn't have a compound statement,
>                    but where would you use it?

But it does have a function:

>>> type("MyClass", (object,), {})
<class '__main__.MyClass'>

Or rather, class is syntactic sugar for the type known as type(), which 
isn't actually a function, but you call it as if it were one.


>     with    ==> I haven't had an occasion to use with,
>                    so can't think of an example where it could
>                    be useful as a compound expression.
>
> Seems to me try-except is kind of sticking out here, and I've
> tried to show what I think are natural and common places where
> a try-except expression would be appropriate.

At the moment, I would vote:

-0 for the general idea
-1 if it allows a bare except


I could possibly be convinced to change my first vote to +0 if I could 
see some real examples of code where this would be useful. (However, I 
don't think there's any power in the universe which could convince me 
that allowing bare excepts here was a good thing :)

Has anyone considered binding to an exception instance? E.g:

    x = f(x) except (ValueError, TypeError) as e: extract_from(e)




-- 
Steven D'Aprano


From masklinn at masklinn.net  Thu Aug 20 16:50:25 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 20 Aug 2009 16:50:25 +0200
Subject: [Python-ideas] exception based conditional expression,
	similar to if-else conditional expression
In-Reply-To: <4A8D5D4E.5090108@lanl.gov>
References: <4A8D5D4E.5090108@lanl.gov>
Message-ID: <EAD0FC33-6DF1-4D9C-8332-07FEB3D013D2@masklinn.net>

On 20 Aug 2009, at 16:27 , Jeff McAninch wrote:
> Seems to me try-except is kind of sticking out here, and I've
> tried to show what I think are natural and common places where
> a try-except expression would be appropriate.
Let me be clear (apparently I wasn't clear enough): I'm not arguing  
against a try/except expression (or, indeed, any expression). I'm  
arguing against creating a brand new expression instead of trying to  
find a way to make the current statements into expressions, which I  
think would be both clearer and better for the language as a whole.


From mcaninch at lanl.gov  Thu Aug 20 16:52:27 2009
From: mcaninch at lanl.gov (Jeff McAninch)
Date: Thu, 20 Aug 2009 08:52:27 -0600
Subject: [Python-ideas] [Fwd: Re: exception based conditional expression,
 similar to if-else conditional expression]
In-Reply-To: <fb6fbf560908200725h7ffc8aeco240aad33390671d5@mail.gmail.com>
References: <4A8D2240.6080107@lanl.gov>
	<50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com>
	<4A8D3112.7030905@lanl.gov>
	<fb6fbf560908200725h7ffc8aeco240aad33390671d5@mail.gmail.com>
Message-ID: <4A8D632B.9010608@lanl.gov>

Jim Jewett wrote:
>
> Probably, but you don't have to use a comprehension.
>
> def g(seq):
>     for e in seq:
>         try:
>             yield float(e)
>         except:
>             pass # whatever, even skipping the element
>
> If you don't need it in generator form, then just collect the results
> into a list and return that instead of yielding.  (And obviously, that
> for loop doesn't even have to be in a separate (generator or) function
> at all.)
>
> -jJ
>   
Certainly there are a number of ways to address this without using an 
exception conditional expression.  But it seems each of these would lead 
to a proliferation of these special functions, as one tries to flexibly 
address the different exception-value pairs that would be applied to 
different operations.

I could for instance define the function Except:

    def Except ( seq, nominal_function, function_lookup ):
        for e in seq:
            try:
               yield nominal_function(e)
            except:
               (exception_type,exception_message) = sys.exc_info()[:2]
               if (exception_type in function_lookup):
                   yield function_lookup[exception_type](e)
               else:
                   raise exception_type, exception_message

Then I could write my simple example as:
    xs = Except( ys, (lambda x: float(x)), {ValueError: (lamba x: 
float('nan'))} )

So I agree, the behaviour can be produced with the language as-is.  But 
many of the evolutions in the language were not put in to address things 
that "can't be done", but rather to let them be done in a more concise, 
robust, and/or elegant way.

-- 
==========================
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: mcaninch at lanl.gov
==========================



From masklinn at masklinn.net  Thu Aug 20 16:54:13 2009
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 20 Aug 2009 16:54:13 +0200
Subject: [Python-ideas] exception based conditional expression,
	similar to if-else conditional expression
In-Reply-To: <200908210025.51640.steve@pearwood.info>
References: <4A8C85CA.6010306@lanl.gov>
	<200908210025.51640.steve@pearwood.info>
Message-ID: <DDC040D5-0169-47AC-A4D0-A9BC133B74CB@masklinn.net>

On 20 Aug 2009, at 16:25 , Steven D'Aprano wrote:
>
> A better way of writing that would be:
>
> x = y.evaluate(locals()) if hasattr(y, 'evaluate') else y
>
> which have the bonus of working correctly even if y gets its  
> evaluate()
> method via inheritance.
>
> Another way, two lines instead of one:
>
> try: x = y.evaluate(locals())
> except AttributeError: x = y
>
>
> A third method: make sure all objects you pass have an evaluate()
> method, even if they are a null-operation that just return self.
>
Fourth method: x = getattr(y, 'evaluate', lambda _: y)(locals())



From greg.ewing at canterbury.ac.nz  Fri Aug 21 00:12:45 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 21 Aug 2009 10:12:45 +1200
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <200908210025.51640.steve@pearwood.info>
References: <4A8C85CA.6010306@lanl.gov>
	<200908210025.51640.steve@pearwood.info>
Message-ID: <4A8DCA5D.6080807@canterbury.ac.nz>

Steven D'Aprano wrote:

> Others have suggested that the colon should be dropped. I dislike that 
> idea, because there's nothing but whitespace delimiting the list of 
> exceptions from the except-expression:
> 
> EXPR except EXCEPT-LIST EXCEPT-EXPR

I agree. Even if it can be parsed unambiguously, it's
going to seem weird and confusing to a human.

So far I haven't seen anything I like better than

   <expr> except <value> if <exception>

despite the fact that it uses the words 'except'
and 'if' in a different way than elsewhere.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Aug 21 00:19:41 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 21 Aug 2009 10:19:41 +1200
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <EAD0FC33-6DF1-4D9C-8332-07FEB3D013D2@masklinn.net>
References: <4A8D5D4E.5090108@lanl.gov>
	<EAD0FC33-6DF1-4D9C-8332-07FEB3D013D2@masklinn.net>
Message-ID: <4A8DCBFD.9050109@canterbury.ac.nz>

Masklinn wrote:
> I'm  
> arguing against creating a brand new expression instead of trying to  
> find a way to make the current statements into expressions, which I  
> think would be both clearer and better for the language as a whole.

Many people have tried, and dismally failed to convince
Guido that their suggested solution would be of benefit
to the language.

-- 
Greg


From ncoghlan at gmail.com  Fri Aug 21 11:49:51 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Aug 2009 19:49:51 +1000
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <200908210046.39235.steve@pearwood.info>
References: <4A8D5D4E.5090108@lanl.gov>
	<200908210046.39235.steve@pearwood.info>
Message-ID: <4A8E6DBF.1010902@gmail.com>

Steven D'Aprano wrote:
> But that one is ugly (that is, I don't think anyone *likes* the syntax, 
> it's just considered the least worst), and some of us consider that its 
> usefulness is dubious, or at least weak. I think the main reason that 
> Python has a ternary if statement is to stop people writing the 
> error-prone:
> 
> cond and x or y
> 
> as a short-circuit ternary if.

That's actually a good point - eliminating the persistent use of buggy
workarounds was definitely one of the major motivations behind accepting
PEP 308.

In this case, there *are* no buggy workarounds: if you want to trap
exception, you have to use a real statement.

Without that additional motivation of bug prevention, I really doubt
this proposal is going to go anywhere.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From stefan_ml at behnel.de  Fri Aug 21 14:16:53 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 21 Aug 2009 14:16:53 +0200
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com>
References: <4A8D2240.6080107@lanl.gov>
	<50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com>
Message-ID: <h6m37l$avk$1@ger.gmane.org>

Chris Rebert wrote:
> On Thu, Aug 20, 2009 at 3:15 AM, Jeff McAninch<mcaninch at lanl.gov> wrote:
>> Calvin Spealman wrote:
>>
>>> -1 on colons in the expression like that. I like the idea of being
>>> able to handle an exception in generator expressions and the like, but
>>> I've never seen a syntax I liked. I think I've favored the idea of
>>> something like `float(x) except float('nan') if ValueError` thinking
>>> it reads more naturally as an expression, puts the real logic
>>> ("convert x to a float or get a NaN float") together, which I think
>>> makes sense.
>>>
>>>
>> Yes, I agree about the colons.  They have no purpose.  I was just blindly
>> following the try-except.  (Duh! on my part)
>>
>> So, in the simple example:
>>  x = float(string) except ValueError float('nan')
> 
> I would feel more comfortable with another keyword in there. Perhaps:
> 
> x = float(string) except ValueError then float('nan')
> 
> Which reads like:
> 
> "x is the float value of string, except in the event of ValueError,
> then it's float('nan')"
> 
> Which I find rather pleasing.

I'm -1 on this. I think it's of much rarer use than even conditional
expressions, has the disadvantage of only handling one special use case
without being easily extensible to more than one exception, and it's easily
served by a simple function that does the same thing (and potentially much
more) but gives it a name.

Stefan



From ilya.nikokoshev at gmail.com  Fri Aug 21 21:51:11 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Fri, 21 Aug 2009 23:51:11 +0400
Subject: [Python-ideas] Interrogate alternate namespace keyword and
	concept
In-Reply-To: <200908151202.29948.steve@pearwood.info>
References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com>
	<200908142117.24072.steve@pearwood.info>
	<d8cc6f9d0908141329w75ae89aesb0a27e598d87bc4b@mail.gmail.com>
	<200908151202.29948.steve@pearwood.info>
Message-ID: <d8cc6f9d0908211251r6ecf47d4l64ab28b0c8b3eb2c@mail.gmail.com>

Thanks for code tips! I appreciate them.

Ok, I kind of not sure I correctly remember why this topic was raised,
so let me remember.
You said that using a separate function has a performance hit. The
purpose of my program was to establish that timing of

function interrogate() == statement interrogate + compile time

It doesn't appear to me that you dispute this claim, so let's accept
it. Since a typical program shouldn't  compile lots of code snippets
(not in number of millions certainly) this means the performance of
function and statement is the same for out purposes. I believe this
closes performance question.


> min(alist) is a more-direct, simpler, faster, easier to read way of
> calculating sorted(alist)[0].

> You're muddying the water by including a for-loop and call to range()
> inside the code snippet being tested. We're trying to compare your
> function interrogate(test, 'value += 1') with the standard call to
> test.value += 1. Why include the time required to generate a range()
> object, and iterate over it ten times, as part of the code snippet? All
> that does is mix up the time required to execute common code and the
> time required to execute the code we care about.
>
> If it's not obvious why your approach is flawed, consider this extreme
> example:
>
> Timer("time.sleep(1000); interrogate(test, 'value += 1')", ...)
> Timer("time.sleep(1000); test.value += 1", ...)
>
> The differences in speed between the interrogate call (using exec) and
> the direct access to test.value will be swamped by the time used by the
> common code.
>
> A more accurate measurement is to remove the "for i in..." part from
> command, and increase the number=1000 argument to Timer.repeat() to
> 10000.
>
>
>> inc5 = compile_command(command)
>
> This is an unfair test. We're comparing directly accessing test.value
> versus indirectly accessing test.value using exec. Regardless of
> whether the caller compiles the statement manually before passing it to
> exec, or just passes it to exec to compile it automatically, the cost
> of that compilation has to be payed. Pulling that outside of the timing
> code just hides the true cost.
>
> Pre-compiling before passing to exec is a good optimization for the
> cases where you need to exec the same code snippet over and over again.
> If you want to execute "for i in range(1000): exec(s)" then it makes
> sense to pull out the compilation of s outside of the loop. But
> generally when timing code snippets, the only purpose of the loop is to
> minimize errors and give more accurate results, so pulling out the
> compilation just hides some of the real cost.
>
>
>> timing("interrogate(test, command)")
>> timing(command.replace('value', 'test.value'))
>> timing("interrogate(test, inc5)")
>>
>> Result:
>>
>> 15
>
> Where does the 15 come from?
>
>> 'interrogate(test, command)' -> 0.0908 ms
>> # test.value = 30015
>> '\nfor i in range(10):\n ? ?test.value += 1\n' -> 0.00408 ms
>> # test.value = 60015
>> 'interrogate(test, inc5)' -> 0.00469 ms
>> # test.value = 90015
>>
>> so interrogate() with additional precompiling introduces very little
>> overhead.
>
> Only because you're ignoring the overhead of pre-compiling. A more
> accurate test would be:
>
> timing("inc5 = compile_command(command); interrogate(test, inc5)")
>
>
>> Though I agree it's inconvenient to write functions as
>> strings;
>
> If you think that's inconvenient, just try writing functions as code
> objects without calling compile :)
>
>
>
> --
> Steven D'Aprano
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From ilya.nikokoshev at gmail.com  Fri Aug 21 22:34:06 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Sat, 22 Aug 2009 00:34:06 +0400
Subject: [Python-ideas] exception based conditional expression,
	similar to 	if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <d8cc6f9d0908211334u6a23b9cdgf60e8b3b953e7632@mail.gmail.com>

First, I'd like to thank you for taking time to write this as a very
concise post. Your idea sounds interesting.

But I don't think I saw any reasonable (= readable) syntax.

Anything with bare semicolon is clearly confusing because bare
semicolon has a well-defined meaning as ending a line and starting new
block.

Anything with bare except clause isn't good because catching all
exceptions is a wrong style.

Anything that encourages the use of \ line continuation character
clearly contradicts the well-established guideline that this character
should be used as a last resort.

-1 on either of above suggestions.


But there are deeper problems even in the "simple example".  To see
them, let me show how this example should look in programming style I
strive for:

def safe_float(string: 'any string', invalid = float('nan')) -> 'float':
    '''Wrapper around float() that converts all invalid values to given param.

    Use with care --- this will also catch invalid inputs and other cases
    where notifying the user with exception is a better idea.

    '''
    try:
        return float(string)
    except ValueError:
        return invalid
...
class tests:
    def testSafeFloat(self):
        assert safe_float('5.7') == float('5.7')
        assert safe_float('invalid') == float('nan')
...

# Here string comes from an experiment and I know it's either empty or should
# be valid.
xs1 = (float(string or 'nan') for string in strings)

# Here string comes from a user input. Tell him if he types something wrong.
for string in strings:
    try:
        xs2[i] = float(string)
    catch ValueError:
        string = input('Please retype string #{}'.format(i))
        ...

# Here strings are a result of series of experiments. Unfortunately we don't
# know what format the researchers used for 'no data' or 'invalid data'. So
# for now we'll just change all of these values to 'nan'.
xs3 = (safe_float(string) for string in strings)


Your proposed solution using exception-based conditional expression:

# Here strings are a result of series of experiments. Unfortunately we don't
# know what format the researchers used for 'no data' or 'invalid data'. So
# for now we'll just change all of these values to 'nan'.
# But, unfortunately, if in the future we'll decide to use some different
# function to process data, we'll have to rewrite this whole cycle...
xs = ((float(string) except ValueError: float('nan')) for string in strings)


I think from my examples you get what I'm trying to say -- when you're
thinking about easily catching exceptions, you're either ok with something
simpler or you're actually trying to apply a function. So why not do it
explicitely?

Ilya.

On Thu, Aug 20, 2009 at 3:07 AM, Jeff McAninch<mcaninch at lanl.gov> wrote:
> I would like to propose an expression, similar to the if-else expression,
> that responds to exceptions.
>
> I had originally posted this (probably mistakenly) on py-dev.? This current
> posting is a cleaned up
> version of the idea, based on responses I got on from the earlier posting.
>
> Abstract:
> Proposal for a conditional expression, similar to the if-else expression,
> that responds to exceptions.
>
> Motivation:
> An expression syntax that responds to exceptions, and which reproduces the
> readability and conciseness of the if-else conditional expression, would
> simplify some exception-handling cases, especially within list
> comprehensions.
>
> Very Simple Example - type coercion:
> Current approach:
> ??? try:
> ??????? x = float(string)
> ??? except:
> ??????? x = float('nan')
>
> Proposed solution using exception-based conditional expression:
> ??? x = float(string) except ValueError: float('nan')
>
>
> Simple Example - type coercion in a list comprehension:
> Current approach:
> ??? def safe_float(string):
> ??????? try:
> ??????????? x = float(string)
> ??????? except ValueError:
> ??????????? x = float('nan')
> ??????? return x
> ??? ...
> ??? xs = (safe(float(string)) for string in strings)
>
> Proposed solution using exception-based conditional expression:
> ??? xs = ((float(string) except ValueError: float('nan')) for string in
> strings)
>
> Discussion:
> In my own python coding, I find I make common use of the if-else conditional
> expression, especially within list comprehensions.? (In one of my packages,
> which has ~5800 lines of code, I found if-else expressions in ~1% of the
> lines.)
>
> Here is a slightly more involved example than the examples presented above.
> In data processing, I often string together a sequence of iterable list
> comprehensions, corresponding to a sequence of operations on a given dataset
> "ys" to produce a processed dataset "x":
> ??? xs = (operation_A(x) for x in ys)
> ??? xs = (operation_B(x) for x in xs if filter_B(x))
> ??? xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in
> xs if filter_C(x))
> ??? # final, explicit list of values
> ??? xs = [ x for x in xs ]
> This is often a flexible way for me to define processing and filtering
> sequences which also seems
> to have good performance on very large datasets.? One advantage is that I
> can quickly mix-and-match from existing processes like this to make a new
> process.? An exception-based conditional would go nicely
> into many of these process sequences, keeping them both robust and flexible.
> ??? xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
>
> I also often have object classes which have some common method or
> attribute.? For instance, some of my objects have scope-dependent values:
> ??? x = y.evaluate(scope))
> where scope is typically locals(), globals(), or some other dictionary-like
> container.? But, to keep my code modular, I want to handle, in the same
> lines of code, objects which do not have some particular method, which leads
> me to lines of code like:
> ??? x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y
> This seems not very "Pythonic", similar to using type-testing instead of
> try-except.? (My impression was that there was a long-standing trend in the
> evolution of Python to remove tests like this, and I thought that was the
> original motivation for the try-except syntax.)
>
> I would much rather write:
> ??? x = y.evaluate(locals()) except AttributeError: y
> or, in the list comprehension example:
> ??? xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
>
> Clearly this can be handled in several ways with the language as it is.? One
> way is to define a new function, as in the second simple example above:
> ??? def safe_evaluate(y,scope):
> ??? ?? try:
> ??? ?? ?? x = y.evaluate(scope)
> ??? ?? except AttributeError:
> ??? ?? ?? x = y
> ??? ?? return x
> ??? ...
> ??? xs = (safe_evaluate(y,locals()) for y in ys)
> but this quickly (in my packages at least) leads to an annoying
> proliferation of "safe_" functions.
> Again, this seems not to be in the "Pythonic" spirit, and is also less
> concise, less readable.? (I also suspect, but have not verified, that this
> is in general less efficient than in-line expressions -- wasn't that part of
> the original motivation for list comprehensions?).
>
> In the thread of my previous post to py-dev, there were comments, questions,
> and suggestions concerning the details of the syntax.? Having reflected on
> this for a couple weeks, I am now most strongly supportive of what is
> essentially just an inline compression of the current try-except syntax.? So
> the following examples would be allowed:
> ??? x = expression0 except: default_expression
> ??? x = expression0 except exception1: expression1 except exception2:
> expression2 except: default_expression
>
> Or, more generally:
> ??? x = expression0\
> ? ? ? ? ? ? except exception1: expression1\
> ??????????? except exception2: expression2\
> ??????????? ...
> ??????????? except exceptionI: expressionI\
> ??????????? ...
> ??????????? except: default_expression
> In this last example, the behaviour would be as follows:
> ??? - evaluate expression0.
> ??????????? If no exception is encountered, return the result.
> ??? - if an exception is encountered,
> ??????????? search for the matching exception in the except clauses.
> ??? - if a matching exception ("exceptionI") is found,
> ??????????? evaluate the corresponding expression ("expressionI"), and
> return the result.
> ??? - if no matching exception is found, and a default except: clause (i.e.,
> one without and exception)
> ? ? ? ??? is given, evaluate default_expression, and return the result.
> ??? - if no matching exception is found, and no default except clause if
> given,
> ????????? pass the exception on to the caller.
> ??? - if a new exception is encountered while evaluating an an except
> expression ("expressionI"),
> ????????? pass the exception on to the caller.
>
> I hope I have made a convincing case here.? This seems to me to be a natural
> ("Pythonic") addition to the language.
>
> Jeff McAninch
>
> --
> ==========================
> Jeffrey E. McAninch, PhD
> Physicist, X-2-IFD
> Los Alamos National Laboratory
> Phone: 505-667-0374
> Email: mcaninch at lanl.gov
> ==========================
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


From rrr at ronadam.com  Sat Aug 22 03:28:04 2009
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 21 Aug 2009 20:28:04 -0500
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <4A8DCA5D.6080807@canterbury.ac.nz>
References: <4A8C85CA.6010306@lanl.gov>	<200908210025.51640.steve@pearwood.info>
	<4A8DCA5D.6080807@canterbury.ac.nz>
Message-ID: <4A8F49A4.4000805@ronadam.com>



Greg Ewing wrote:
> Steven D'Aprano wrote:
> 
>> Others have suggested that the colon should be dropped. I dislike that 
>> idea, because there's nothing but whitespace delimiting the list of 
>> exceptions from the except-expression:
>>
>> EXPR except EXCEPT-LIST EXCEPT-EXPR
> 
> I agree. Even if it can be parsed unambiguously, it's
> going to seem weird and confusing to a human.
> 
> So far I haven't seen anything I like better than
> 
>   <expr> except <value> if <exception>
> 
> despite the fact that it uses the words 'except'
> and 'if' in a different way than elsewhere.


Possibly the only way to do this within those constraints is a purely 
syntactic.

     value = (expr1 :exception: expr2)

Sense most people (and editors) recognize exceptions names easily, there 
really isn't a strong need to use the key word "except" in an expression 
like this.

The colons in this case can be considered to be similar to slice syntax. 
And if the colons are still considered bad, possibly some other symbol 
could be used.

Ron









From steve at pearwood.info  Sat Aug 22 07:32:14 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 22 Aug 2009 15:32:14 +1000
Subject: [Python-ideas] exception based conditional expression,
	similar to if-else conditional expression
In-Reply-To: <4A8F49A4.4000805@ronadam.com>
References: <4A8C85CA.6010306@lanl.gov> <4A8DCA5D.6080807@canterbury.ac.nz>
	<4A8F49A4.4000805@ronadam.com>
Message-ID: <200908221532.15272.steve@pearwood.info>

On Sat, 22 Aug 2009 11:28:04 am Ron Adam wrote:
> Greg Ewing wrote:
> > Steven D'Aprano wrote:
> >> Others have suggested that the colon should be dropped. I dislike
> >> that idea, because there's nothing but whitespace delimiting the
> >> list of exceptions from the except-expression:
> >>
> >> EXPR except EXCEPT-LIST EXCEPT-EXPR
> >
> > I agree. Even if it can be parsed unambiguously, it's
> > going to seem weird and confusing to a human.
> >
> > So far I haven't seen anything I like better than
> >
> >   <expr> except <value> if <exception>
> >
> > despite the fact that it uses the words 'except'
> > and 'if' in a different way than elsewhere.
>
> Possibly the only way to do this within those constraints 

Which constraints? Are you referring to the fact that any proposed 
syntax must be capable of being unambiguously parsed?


> is a purely syntactic.

I'm not sure I understand. Can you give an example of a proposal which 
*isn't* syntactic?

>      value = (expr1 :exception: expr2)

Are the brackets intended to be part of the syntax, or are they there 
just for grouping? In other words, would this be allowed?

value = function(expr1 :exception: expr2)

Or would we have to write this?

value = function((expr1 :exception: expr2))


> Sense most people (and editors) recognize exceptions names easily,
> there really isn't a strong need to use the key word "except" in an
> expression like this.

The word "except" cues in the reader that they are seeing an 
exception-expression, rather than some sort of data structure like 
{key:value} or (a, tuple).

For the same reason, list comprehensions use "for":

[x+1 for x in alist]

rather than mathematical notation:

{x+1 ? x ? alist}

(if that doesn't display for you, the first odd character is an upside 
down A, "for all", and the second is a curved E, "element of"). If you 
don't like unicode-based source code, we could replace the "for all" 
and "element of" symbols with colons:

[x+1 : x : alist]

but we don't because the words "for" and "in" give the reader hints as 
to what is being done, which a colon would not.

Words are just symbols, so anytime you have syntax including a keyword, 
you could replace the keyword with a symbol. That way leads to APL. As 
a general rule, keywords are more readable, punctuation is more 
concise.



-- 
Steven D'Aprano


From ilya.nikokoshev at gmail.com  Sat Aug 22 11:36:26 2009
From: ilya.nikokoshev at gmail.com (ilya)
Date: Sat, 22 Aug 2009 13:36:26 +0400
Subject: [Python-ideas] exception based conditional expression,
	similar to 	if-else conditional expression
In-Reply-To: <4A8C85CA.6010306@lanl.gov>
References: <4A8C85CA.6010306@lanl.gov>
Message-ID: <d8cc6f9d0908220236i3e2c3656sdf9241e77978260d@mail.gmail.com>

I like your use case, but it appears to me a processing task that
repeats a lot deserves its own function, so:

xs = apply(operation_N, {exceptionN: operation_Nprime}, xs)

which also happens to be shorter than

xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)

Similarly, multiline try/except as in your last example

    x = expression0\
            except exception1: expression1\
            except exception2: expression2\
            ...
            except exceptionI: expressionI\
            ...
            except: default_expression

is certainly useful. But we're lucky, nearly identical syntax already
exists out of the box!

    try: _ = expression0
    except exception1: _ = expression1
    except exception2: _ = expression2
    ...
    except exceptionI: _ = expressionI
    ...
    except: _ = default_expression
    x = _

What's so good about the your snippet that is bad about mine? Anyway,
instead of either of those I would prefer

    # In most cases expression0 returns well-formed foo.
    try: _ = expression0
    except exception1: # Take care of the case of bar being not frobnicable.
        _ = expression1
    except exception2:
        # This looks like magic, but is isn't. See documentation for
        # exception2 at http:// some_url for explanation.
        _ = expression2
    ...
    except exceptionI: _ = expressionI
    ...
    except exceptionLast:
        _ = expressionLast
    # All cases I could think of are exhausted! Of course, anything can
    # happen so any other exception will propagate up.
    x = _


On Thu, Aug 20, 2009 at 3:07 AM, Jeff McAninch<mcaninch at lanl.gov> wrote:
> I would like to propose an expression, similar to the if-else expression,
> that responds to exceptions.
>
> I had originally posted this (probably mistakenly) on py-dev.? This current
> posting is a cleaned up
> version of the idea, based on responses I got on from the earlier posting.
>
> Abstract:
> Proposal for a conditional expression, similar to the if-else expression,
> that responds to exceptions.
>
> Motivation:
> An expression syntax that responds to exceptions, and which reproduces the
> readability and conciseness of the if-else conditional expression, would
> simplify some exception-handling cases, especially within list
> comprehensions.
>
> Very Simple Example - type coercion:
> Current approach:
> ??? try:
> ??????? x = float(string)
> ??? except:
> ??????? x = float('nan')
>
> Proposed solution using exception-based conditional expression:
> ??? x = float(string) except ValueError: float('nan')
>
>
> Simple Example - type coercion in a list comprehension:
> Current approach:
> ??? def safe_float(string):
> ??????? try:
> ??????????? x = float(string)
> ??????? except ValueError:
> ??????????? x = float('nan')
> ??????? return x
> ??? ...
> ??? xs = (safe(float(string)) for string in strings)
>
> Proposed solution using exception-based conditional expression:
> ??? xs = ((float(string) except ValueError: float('nan')) for string in
> strings)
>
> Discussion:
> In my own python coding, I find I make common use of the if-else conditional
> expression, especially within list comprehensions.? (In one of my packages,
> which has ~5800 lines of code, I found if-else expressions in ~1% of the
> lines.)
>
> Here is a slightly more involved example than the examples presented above.
> In data processing, I often string together a sequence of iterable list
> comprehensions, corresponding to a sequence of operations on a given dataset
> "ys" to produce a processed dataset "x":
> ??? xs = (operation_A(x) for x in ys)
> ??? xs = (operation_B(x) for x in xs if filter_B(x))
> ??? xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in
> xs if filter_C(x))
> ??? # final, explicit list of values
> ??? xs = [ x for x in xs ]
> This is often a flexible way for me to define processing and filtering
> sequences which also seems
> to have good performance on very large datasets.? One advantage is that I
> can quickly mix-and-match from existing processes like this to make a new
> process.? An exception-based conditional would go nicely
> into many of these process sequences, keeping them both robust and flexible.
> ??? xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs)
>
> I also often have object classes which have some common method or
> attribute.? For instance, some of my objects have scope-dependent values:
> ??? x = y.evaluate(scope))
> where scope is typically locals(), globals(), or some other dictionary-like
> container.? But, to keep my code modular, I want to handle, in the same
> lines of code, objects which do not have some particular method, which leads
> me to lines of code like:
> ??? x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y
> This seems not very "Pythonic", similar to using type-testing instead of
> try-except.? (My impression was that there was a long-standing trend in the
> evolution of Python to remove tests like this, and I thought that was the
> original motivation for the try-except syntax.)
>
> I would much rather write:
> ??? x = y.evaluate(locals()) except AttributeError: y
> or, in the list comprehension example:
> ??? xs = (y.evaluate(locals()) except AttributeError: y for y in ys)
>
> Clearly this can be handled in several ways with the language as it is.? One
> way is to define a new function, as in the second simple example above:
> ??? def safe_evaluate(y,scope):
> ??? ?? try:
> ??? ?? ?? x = y.evaluate(scope)
> ??? ?? except AttributeError:
> ??? ?? ?? x = y
> ??? ?? return x
> ??? ...
> ??? xs = (safe_evaluate(y,locals()) for y in ys)
> but this quickly (in my packages at least) leads to an annoying
> proliferation of "safe_" functions.
> Again, this seems not to be in the "Pythonic" spirit, and is also less
> concise, less readable.? (I also suspect, but have not verified, that this
> is in general less efficient than in-line expressions -- wasn't that part of
> the original motivation for list comprehensions?).
>
> In the thread of my previous post to py-dev, there were comments, questions,
> and suggestions concerning the details of the syntax.? Having reflected on
> this for a couple weeks, I am now most strongly supportive of what is
> essentially just an inline compression of the current try-except syntax.? So
> the following examples would be allowed:
> ??? x = expression0 except: default_expression
> ??? x = expression0 except exception1: expression1 except exception2:
> expression2 except: default_expression
>
> Or, more generally:
> ??? x = expression0\
> ? ? ? ? ? ? except exception1: expression1\
> ??????????? except exception2: expression2\
> ??????????? ...
> ??????????? except exceptionI: expressionI\
> ??????????? ...
> ??????????? except: default_expression
> In this last example, the behaviour would be as follows:
> ??? - evaluate expression0.
> ??????????? If no exception is encountered, return the result.
> ??? - if an exception is encountered,
> ??????????? search for the matching exception in the except clauses.
> ??? - if a matching exception ("exceptionI") is found,
> ??????????? evaluate the corresponding expression ("expressionI"), and
> return the result.
> ??? - if no matching exception is found, and a default except: clause (i.e.,
> one without and exception)
> ? ? ? ??? is given, evaluate default_expression, and return the result.
> ??? - if no matching exception is found, and no default except clause if
> given,
> ????????? pass the exception on to the caller.
> ??? - if a new exception is encountered while evaluating an an except
> expression ("expressionI"),
> ????????? pass the exception on to the caller.
>
> I hope I have made a convincing case here.? This seems to me to be a natural
> ("Pythonic") addition to the language.
>
> Jeff McAninch
>
> --
> ==========================
> Jeffrey E. McAninch, PhD
> Physicist, X-2-IFD
> Los Alamos National Laboratory
> Phone: 505-667-0374
> Email: mcaninch at lanl.gov
> ==========================
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>


From rrr at ronadam.com  Sat Aug 22 18:29:19 2009
From: rrr at ronadam.com (Ron Adam)
Date: Sat, 22 Aug 2009 11:29:19 -0500
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <200908221532.15272.steve@pearwood.info>
References: <4A8C85CA.6010306@lanl.gov>
	<4A8DCA5D.6080807@canterbury.ac.nz>	<4A8F49A4.4000805@ronadam.com>
	<200908221532.15272.steve@pearwood.info>
Message-ID: <4A901CDF.8080908@ronadam.com>



Steven D'Aprano wrote:
> On Sat, 22 Aug 2009 11:28:04 am Ron Adam wrote:
>> Greg Ewing wrote:
>>> Steven D'Aprano wrote:
>>>> Others have suggested that the colon should be dropped. I dislike
>>>> that idea, because there's nothing but whitespace delimiting the
>>>> list of exceptions from the except-expression:
>>>>
>>>> EXPR except EXCEPT-LIST EXCEPT-EXPR
>>> I agree. Even if it can be parsed unambiguously, it's
>>> going to seem weird and confusing to a human.
>>>
>>> So far I haven't seen anything I like better than
>>>
>>>   <expr> except <value> if <exception>
>>>
>>> despite the fact that it uses the words 'except'
>>> and 'if' in a different way than elsewhere.
>> Possibly the only way to do this within those constraints 
> 
> Which constraints? Are you referring to the fact that any proposed 
> syntax must be capable of being unambiguously parsed?

And also to not using 'except' and 'if' in a different way than elsewhere.


>> is a purely syntactic.
> 
> I'm not sure I understand. Can you give an example of a proposal which 
> *isn't* syntactic?

Technically you are correct, but I was trying to refer to patterns that do 
not use keywords to identify the operation.  For example slice operations 
use a pattern rather than a unique symbol to identify the operation.

So this is an example of a pattern that could work.

>>      value = (expr1 :exception: expr2)

> Are the brackets intended to be part of the syntax, or are they there 
> just for grouping? In other words, would this be allowed?

Yes, I intended the brackets to be part of the pattern.


> value = function(expr1 :exception: expr2)
> 
> Or would we have to write this?
> 
> value = function((expr1 :exception: expr2))

I supose it could work like generator expressions do.  Then both of these 
could be equivalent.


>> Sense most people (and editors) recognize exceptions names easily,
>> there really isn't a strong need to use the key word "except" in an
>> expression like this.
> 
> The word "except" cues in the reader that they are seeing an 
> exception-expression, rather than some sort of data structure like 
> {key:value} or (a, tuple).

True, it needs to be unique enough to be easily identifiable or it would 
not be worth it.


> For the same reason, list comprehensions use "for":
> 
> [x+1 for x in alist]
> 
> rather than mathematical notation:
> 
> {x+1 ? x ? alist}

It displayed fine.  :-)

> (if that doesn't display for you, the first odd character is an upside 
> down A, "for all", and the second is a curved E, "element of"). If you 
> don't like unicode-based source code, we could replace the "for all" 
> and "element of" symbols with colons:
> 
> [x+1 : x : alist]

I think having the square brackets with the colons would be a bit too close 
to slice notation here.  ie... it's not unique enough to be easily 
identifiable.


> but we don't because the words "for" and "in" give the reader hints as 
> to what is being done, which a colon would not.

This is especially true for more complex operations such as looping and 
repeating.


> Words are just symbols, so anytime you have syntax including a keyword, 
> you could replace the keyword with a symbol. That way leads to APL. As 
> a general rule, keywords are more readable, punctuation is more 
> concise.

Yes, I agree, but python isn't that strict about everything.  To me a well 
defined exception expression that



All the previous suggestions view the term as needing three values, so you 
need two separators along with unique symbols (or keywords) to come up with 
something that fits the need in a clear and concise way.  I was thinking 
what if we could use the exception object itself as an operator, then the 
term becomes even simpler and more concise.  But the only way I can think 
to do that in a clear way is to use syntax to identify the exception, such 
as putting colons, or some other symbol around the exception.  <shrug>


Ron




























From rrr at ronadam.com  Sat Aug 22 21:53:33 2009
From: rrr at ronadam.com (Ron Adam)
Date: Sat, 22 Aug 2009 14:53:33 -0500
Subject: [Python-ideas] exception based conditional expression,
 similar to if-else conditional expression
In-Reply-To: <4A901CDF.8080908@ronadam.com>
References: <4A8C85CA.6010306@lanl.gov>	<4A8DCA5D.6080807@canterbury.ac.nz>	<4A8F49A4.4000805@ronadam.com>	<200908221532.15272.steve@pearwood.info>
	<4A901CDF.8080908@ronadam.com>
Message-ID: <4A904CBD.6080601@ronadam.com>



Ron Adam wrote:
> 
> 
> Steven D'Aprano wrote:

>> Words are just symbols, so anytime you have syntax including a 
>> keyword, you could replace the keyword with a symbol. That way leads 
>> to APL. As a general rule, keywords are more readable, punctuation is 
>> more concise.
> 
> Yes, I agree, but python isn't that strict about everything.  To me a 
> well defined exception expression that

Editing fragment...  Disregard the incomplete sentence.


> All the previous suggestions view the term as needing three values, so 
> you need two separators along with unique symbols (or keywords) to come 
> up with something that fits the need in a clear and concise way.  I was 
> thinking what if we could use the exception object itself as an 
> operator, then the term becomes even simpler and more concise.  But the 
> only way I can think to do that in a clear way is to use syntax to 
> identify the exception, such as putting colons, or some other symbol 
> around the exception.  <shrug>

An additional note, the when you view an exception object in the console it 
is bracketed by the less than and and greater than signs.  Maybe there is a 
way to use those in a manner that is consistent with the string repr() returns.

Ron









From cmjohnson.mailinglist at gmail.com  Sun Aug 23 00:23:04 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Sat, 22 Aug 2009 12:23:04 -1000
Subject: [Python-ideas] exception based conditional expression,
	similar to 	if-else conditional expression
In-Reply-To: <4A904CBD.6080601@ronadam.com>
References: <4A8C85CA.6010306@lanl.gov> <4A8DCA5D.6080807@canterbury.ac.nz> 
	<4A8F49A4.4000805@ronadam.com> <200908221532.15272.steve@pearwood.info>
	<4A901CDF.8080908@ronadam.com> <4A904CBD.6080601@ronadam.com>
Message-ID: <3bdda690908221523o68fcae55n9a9f47ac536e8ea6@mail.gmail.com>

Ron Adam wrote:

> An additional note, the when you view an exception object in the console it
> is bracketed by the less than and and greater than signs. ?Maybe there is a
> way to use those in a manner that is consistent with the string repr()
> returns.

There won't be unless you rule out using "x > y" as an expression that
gives the default value. But doing that will make Python's grammar way
too complicated, and you're still left with the problem of "x, y = a <
b, b > c" which is legal today.

It's my understanding that Ruby has run into a lot of ambiguity with
its use of | as both a bitwise operator and a part of a block
definition. This seems worse.


From steve at pearwood.info  Sun Aug 23 03:39:54 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 23 Aug 2009 11:39:54 +1000
Subject: [Python-ideas] exception based conditional expression,
	similar to if-else conditional expression
In-Reply-To: <4A904CBD.6080601@ronadam.com>
References: <4A8C85CA.6010306@lanl.gov> <4A901CDF.8080908@ronadam.com>
	<4A904CBD.6080601@ronadam.com>
Message-ID: <200908231139.55627.steve@pearwood.info>

On Sun, 23 Aug 2009 05:53:33 am Ron Adam wrote:
> An additional note, the when you view an exception object in the
> console it is bracketed by the less than and and greater than signs.
> ?Maybe there is a way to use those in a manner that is consistent
> with the string repr() returns.

Lots of things have a repr() which includes < and > signs, including 
repr() itself.

>>> print repr
<built-in function repr>

Also new style classes and instances, types, functions, methods, etc. 
Exceptions are printed with < > signs because exceptions are subclasses 
of object. There's nothing specifically "exception-like" about the use 
of < and > in the repr of exceptions.


-- 
Steven D'Aprano


From travis+ml-python-ideas at subspacefield.org  Mon Aug 24 22:14:58 2009
From: travis+ml-python-ideas at subspacefield.org (travis+ml-python-ideas at subspacefield.org)
Date: Mon, 24 Aug 2009 15:14:58 -0500
Subject: [Python-ideas] expose {set,get}res{uid,gid} in posixmodule.c
Message-ID: <20090824201458.GA6647@subspacefield.org>

So there has been a little discussion on the python mailing list regarding
a request I had to expose {set,get}res{uid,gid}.

The general consensus there is that even though it isn't a POSIX standard,
it is shared among many Unix OSes and belongs in the same module as setuid
and friends, namely os, which maps to posixmodule.c

Previous discusion and background as to why this is desirable:
http://bugs.python.org/issue6508

If this is the Right Way to do it, I'll brush up on autotools and see
about submitting a patch...
-- 
Obama Nation | My emails do not have attachments; it's a digital signature
that your mail program doesn't understand. | http://www.subspacefield.org/~travis/ 
If you are a spammer, please email john at subspacefield.org to get blacklisted.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090824/6dfb4235/attachment.pgp>

From dickinsm at gmail.com  Tue Aug 25 16:25:23 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 25 Aug 2009 15:25:23 +0100
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
	operations on bytes)
In-Reply-To: <acd65fa20908161144m14a04044pd8b9100880cdc0c1@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>
	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>
	<4A801D0C.2000502@gmail.com>
	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>
	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>
	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>
	<5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com>
	<acd65fa20908150948k403c52d0t99b80ae20656546b@mail.gmail.com>
	<5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com>
	<acd65fa20908161144m14a04044pd8b9100880cdc0c1@mail.gmail.com>
Message-ID: <5c6f2a5d0908250725yc1d1808p60ae4711f47ffe1b@mail.gmail.com>

[Nick Coghlan]
> What would we be converting them to in that case? 2.x strings? (I don't
> have a problem with that, just pointing out there may be some additional
> work due to changing the target type).

Yes, the 2.x 'str' type.  I haven't thought about whether this could cause
2-to-3 translation problems for people using this in 2.x.  (I don't
immediately see why it would.)

Might there be political reasons not to backport this to 2.x?  I seem to
recall it being suggested at the PyCon language summit that we
should consider making new features 3.x only, but I don't entirely
remember what the rationale for this was.

[Alexandre Vassalotti]
> On Sat, Aug 15, 2009 at 2:42 PM, Mark Dickinson<dickinsm at gmail.com> wrote:
>> If these additions to int go in, then presumably the _PyLong_AsBytes
>> and _PyLong_FromBytes functions should be documented and made public
>> (and have their leading underscores removed, too).
>
> You are referring to _PyLong_FromByteArray and _PyLong_AsByteArray, right?

Whoops!  Yes, that's what I meant.  Thanks.  :)

Sorry for my silence on the issue tracker.  I'll try to find time to look at
your new patch this weekend.

Mark


From ncoghlan at gmail.com  Wed Aug 26 13:13:33 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 26 Aug 2009 21:13:33 +1000
Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise
 operations on bytes)
In-Reply-To: <5c6f2a5d0908250725yc1d1808p60ae4711f47ffe1b@mail.gmail.com>
References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com>	<42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com>	<4A801D0C.2000502@gmail.com>	<5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com>	<42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com>	<acd65fa20908150845v21402c17p790f9821a2e69cba@mail.gmail.com>	<5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com>	<acd65fa20908150948k403c52d0t99b80ae20656546b@mail.gmail.com>	<5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com>	<acd65fa20908161144m14a04044pd8b9100880cdc0c1@mail.gmail.com>
	<5c6f2a5d0908250725yc1d1808p60ae4711f47ffe1b@mail.gmail.com>
Message-ID: <4A9518DD.8010506@gmail.com>

Mark Dickinson wrote:
> [Nick Coghlan]
>> What would we be converting them to in that case? 2.x strings? (I don't
>> have a problem with that, just pointing out there may be some additional
>> work due to changing the target type).
> 
> Yes, the 2.x 'str' type.  I haven't thought about whether this could cause
> 2-to-3 translation problems for people using this in 2.x.  (I don't
> immediately see why it would.)
> 
> Might there be political reasons not to backport this to 2.x?  I seem to
> recall it being suggested at the PyCon language summit that we
> should consider making new features 3.x only, but I don't entirely
> remember what the rationale for this was.

I wasn't there, but there were rumbles about having a few nice carrots
in 3.x to help people think it was worthwhile to switch. That said, I
think the final consensus was that new features *must* go into 3.x, but
if they can be backported and someone is happy to do the work then
backporting isn't an issue.

And in this case, the fact that the methods will produce/consume strings
in 2.x shouldn't cause any more problems than any other cases that
involve 2.x storing binary data in text strings.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From solipsis at pitrou.net  Thu Aug 27 06:13:02 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 27 Aug 2009 04:13:02 +0000 (UTC)
Subject: [Python-ideas]
	=?utf-8?b?ZXhwb3NlIHtzZXQsZ2V0fXJlc3t1aWQsZ2lkfSBp?=
	=?utf-8?q?n_posixmodule=2Ec?=
References: <20090824201458.GA6647@subspacefield.org>
Message-ID: <loom.20090827T061200-237@post.gmane.org>

<travis+ml-python-ideas at ...> writes:
> 
> So there has been a little discussion on the python mailing list regarding
> a request I had to expose {set,get}res{uid,gid}.
> 
> The general consensus there is that even though it isn't a POSIX standard,
> it is shared among many Unix OSes and belongs in the same module as setuid
> and friends, namely os, which maps to posixmodule.c

It seems reasonable indeed.

> Previous discusion and background as to why this is desirable:
> http://bugs.python.org/issue6508
> 
> If this is the Right Way to do it, I'll brush up on autotools and see
> about submitting a patch...

Please do!
Also, once a patch has been submitted, further discussion and review should take
place on the aforementioned tracker issue.

Regards

Antoine.




From ubershmekel at gmail.com  Mon Aug 31 02:45:51 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Mon, 31 Aug 2009 03:45:51 +0300
Subject: [Python-ideas] Add a builtin method to 'int' for base/radix
	conversion
Message-ID: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>

I believe int(s, base) needs an inverse function to allow string
representation with different bases. An example use case is 'hashing' a
counter like video ID's on youtube, you could use a regular int internally
and publish a shorter base-62 id
for links.

This subject was discussed 2.5 years ago:
http://mail.python.org/pipermail/python-dev/2006-January/059789.html

I opened a feature request ticket:
http://bugs.python.org/issue6783

Some of the questions that remain:
1. Whether this should be a method for int or a regular function in a
standard library module like math.
2. What should the method/function be called? (base_convert, radix, etc)

What do you guys think?


RunThePun
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090831/739cc44b/attachment.html>

From cmjohnson.mailinglist at gmail.com  Mon Aug 31 06:49:53 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Sun, 30 Aug 2009 18:49:53 -1000
Subject: [Python-ideas] Add a builtin method to 'int' for base/radix
	conversion
In-Reply-To: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
Message-ID: <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com>

How would it work for bases beyond 10? OK, so base-16 has a well known
encoding, but what about base-100? Base-16 is case insensitive, but
would Base-37 be case insensitive??

ISTM that the common encodings of bin, oct, and hex are already
covered by built-in functions and for bases beyond that, you have
decide what system to use for encoding symbols that are out of range.
Do people really need trinary through septary that often? Is there a
single, best-practices way of doing base-1013? Feels like YAGNI to me,
but I could be wrong.

? Carl Johnson


From cmjohnson.mailinglist at gmail.com  Mon Aug 31 06:55:03 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Sun, 30 Aug 2009 18:55:03 -1000
Subject: [Python-ideas] Add a builtin method to 'int' for base/radix
	conversion
In-Reply-To: <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> 
	<3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com>
Message-ID: <3bdda690908302155v145d7578ua9e994db208327a7@mail.gmail.com>

Doing some testing with int, I found:

ValueError: int() base must be >= 2 and <= 36

That resolves a lot of questions (it has to be case insensitive), but
takes away ability to do the motivating use case: writing out video
ids in Base-62.

?Carl

On Sun, Aug 30, 2009 at 6:49 PM, Carl
Johnson<cmjohnson.mailinglist at gmail.com> wrote:
> How would it work for bases beyond 10? OK, so base-16 has a well known
> encoding, but what about base-100? Base-16 is case insensitive, but
> would Base-37 be case insensitive??
>
> ISTM that the common encodings of bin, oct, and hex are already
> covered by built-in functions and for bases beyond that, you have
> decide what system to use for encoding symbols that are out of range.
> Do people really need trinary through septary that often? Is there a
> single, best-practices way of doing base-1013? Feels like YAGNI to me,
> but I could be wrong.
>
> ? Carl Johnson
>


From greg at krypto.org  Mon Aug 31 08:53:12 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 30 Aug 2009 23:53:12 -0700
Subject: [Python-ideas] Add a builtin method to 'int' for base/radix
	conversion
In-Reply-To: <3bdda690908302155v145d7578ua9e994db208327a7@mail.gmail.com>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> 
	<3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com> 
	<3bdda690908302155v145d7578ua9e994db208327a7@mail.gmail.com>
Message-ID: <52dc1c820908302353t5ddf8012u98dd6ee2b784bfc5@mail.gmail.com>

On Sun, Aug 30, 2009 at 9:55 PM, Carl Johnson <
cmjohnson.mailinglist at gmail.com> wrote:

> Doing some testing with int, I found:
>
> ValueError: int() base must be >= 2 and <= 36
>
> That resolves a lot of questions (it has to be case insensitive), but
> takes away ability to do the motivating use case: writing out video
> ids in Base-62.
>

If there are widely accepted and used non-controversial standard character
-> digit mappings for bases greater than that I don't immediately see any
reasons not to support them for ints but I do not think we should go beyond
base 255.

If someone wants base 256 they should use struct.pack() and beyond that I
expect to find people fighting over encoding schemes. ;)

Since this is the ideas list...  Consider adding a digit character sequence
or map passed to the conversion function defining which characters map to
which digit number.



>
> ?Carl
>
> On Sun, Aug 30, 2009 at 6:49 PM, Carl
> Johnson<cmjohnson.mailinglist at gmail.com> wrote:
> > How would it work for bases beyond 10? OK, so base-16 has a well known
> > encoding, but what about base-100? Base-16 is case insensitive, but
> > would Base-37 be case insensitive??
> >
> > ISTM that the common encodings of bin, oct, and hex are already
> > covered by built-in functions and for bases beyond that, you have
> > decide what system to use for encoding symbols that are out of range.
> > Do people really need trinary through septary that often? Is there a
> > single, best-practices way of doing base-1013? Feels like YAGNI to me,
> > but I could be wrong.
> >
> > ? Carl Johnson
> >
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20090830/90a20973/attachment.html>

From greg.ewing at canterbury.ac.nz  Mon Aug 31 10:21:45 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 31 Aug 2009 20:21:45 +1200
Subject: [Python-ideas] Add a builtin method to 'int' for base/radix
 conversion
In-Reply-To: <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
	<3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com>
Message-ID: <4A9B8819.6000107@canterbury.ac.nz>

Carl Johnson wrote:
> How would it work for bases beyond 10?

If you keep going through the alphabet you can get up
to base 36. If anyone cares beyond that, it could take
a string of characters to use for digits. With Unicode
that would provide quite a wide range. :-)

-- 
Greg


From masklinn at masklinn.net  Mon Aug 31 10:32:44 2009
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 31 Aug 2009 10:32:44 +0200
Subject: [Python-ideas] Add a builtin method to 'int' for base/radix
	conversion
In-Reply-To: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
Message-ID: <31909350-5278-4B75-B9B6-081D6364FF94@masklinn.net>

On 31 Aug 2009, at 02:45 , Yuvgoog Greenle wrote:
> Some of the questions that remain:
> 1. Whether this should be a method for int or a regular function in a
> standard library module like math.
> 2. What should the method/function be called? (base_convert, radix,  
> etc)
>
> What do you guys think?
The format mini-language already has presentation types for binary,  
octal, dec and hex outputs. Maybe this could be expanded in some way  
to get arbitrary base output (between 2 and 36 to match int())


From ncoghlan at gmail.com  Mon Aug 31 15:00:27 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 31 Aug 2009 23:00:27 +1000
Subject: [Python-ideas] Add a builtin method to 'int' for
	base/radix	conversion
In-Reply-To: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
Message-ID: <4A9BC96B.2010207@gmail.com>

Yuvgoog Greenle wrote:
> I believe int(s, base) needs an inverse function to allow string
> representation with different bases. An example use case is 'hashing' a
> counter like video ID's on youtube, you could use a regular int
> internally and publish a shorter base-62 id 
> for links.
> 
> This subject was discussed 2.5 years ago:
> http://mail.python.org/pipermail/python-dev/2006-January/059789.html
> 
> I opened a feature request ticket:
> http://bugs.python.org/issue6783
> 
> Some of the questions that remain:
> 1. Whether this should be a method for int or a regular function in a
> standard library module like math.
> 2. What should the method/function be called? (base_convert, radix, etc)
> 
> What do you guys think?

This has been coming up for years and always gets bogged down in a
spelling argument (a method on int, a function in the math module and an
update to the str.format mini language would be the current contenders).

However, most of the actual real use cases for bases between 2 and 36
were dealt with by the addition of binary and octal output to string
formatting so the impetus to do anything about it is now a lot lower.

As far as bases between 37 and 62 go, that would involve first getting
agreement on extending int() to handle those bases by allowing case
sensitive digit parsing. Presumably that would use string lexical
ordering so that int('a', 37) > int('A', 37) and int('b', 37) would
raise an exception.

That would only be intuitive to someone that knows how ASCII based
alphanumeric ordering works though. Probably not an impossible sell, but
you would want a compelling use case (converting long strings of digits
into shorter link URLs is a pretty good one though - I've seen that in
places other than youtube).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From masklinn at masklinn.net  Mon Aug 31 15:58:53 2009
From: masklinn at masklinn.net (Masklinn)
Date: Mon, 31 Aug 2009 15:58:53 +0200
Subject: [Python-ideas] Add a builtin method to 'int' for
	base/radix	conversion
In-Reply-To: <4A9BC96B.2010207@gmail.com>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>
	<4A9BC96B.2010207@gmail.com>
Message-ID: <1AD8D421-A250-421D-A6ED-4992234A5B96@masklinn.net>

On 31 Aug 2009, at 15:00 , Nick Coghlan wrote:
Yuvgoog Greenle wrote:
>> I believe int(s, base) needs an inverse function to allow string
>> representation with different bases. An example use case is  
>> 'hashing' a
>> counter like video ID's on youtube, you could use a regular int
>> internally and publish a shorter base-62 id
>> for links.
>>
>> This subject was discussed 2.5 years ago:
>> http://mail.python.org/pipermail/python-dev/2006-January/059789.html
>>
>> I opened a feature request ticket:
>> http://bugs.python.org/issue6783
>>
>> Some of the questions that remain:
>> 1. Whether this should be a method for int or a regular function in a
>> standard library module like math.
>> 2. What should the method/function be called? (base_convert, radix,  
>> etc)
>>
>> What do you guys think?
>
> This has been coming up for years and always gets bogged down in a
> spelling argument (a method on int, a function in the math module  
> and an
> update to the str.format mini language would be the current  
> contenders).
>
> However, most of the actual real use cases for bases between 2 and 36
> were dealt with by the addition of binary and octal output to string
> formatting so the impetus to do anything about it is now a lot lower.
>
> As far as bases between 37 and 62 go, that would involve first getting
> agreement on extending int() to handle those bases by allowing case
> sensitive digit parsing. Presumably that would use string lexical
> ordering so that int('a', 37) > int('A', 37) and int('b', 37) would
> raise an exception.
>
> That would only be intuitive to someone that knows how ASCII based
> alphanumeric ordering works though.
Or it could be handled via a translation table (needed both ways of  
course) mapping n indexes to n characters (with n the base you're  
working with), defaulting to something sane.

Though I'm not sure this is of much interest really: even Erlang  
(which provides pretty good base conversion tools: it supports literal  
integers of any base between 2 and 36) doesn't natively support bases  
beyond 36. A library would probably be better for those more  
conflictual (or less intuitive) ranges.



From python at mrabarnett.plus.com  Mon Aug 31 17:47:24 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 31 Aug 2009 16:47:24 +0100
Subject: [Python-ideas] Add a builtin method to 'int'
	for	base/radix	conversion
In-Reply-To: <1AD8D421-A250-421D-A6ED-4992234A5B96@masklinn.net>
References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com>	<4A9BC96B.2010207@gmail.com>
	<1AD8D421-A250-421D-A6ED-4992234A5B96@masklinn.net>
Message-ID: <4A9BF08C.9060607@mrabarnett.plus.com>

Masklinn wrote:
> On 31 Aug 2009, at 15:00 , Nick Coghlan wrote:
> Yuvgoog Greenle wrote:
>>> I believe int(s, base) needs an inverse function to allow string
>>> representation with different bases. An example use case is 'hashing' a
>>> counter like video ID's on youtube, you could use a regular int
>>> internally and publish a shorter base-62 id
>>> for links.
>>>
>>> This subject was discussed 2.5 years ago:
>>> http://mail.python.org/pipermail/python-dev/2006-January/059789.html
>>>
>>> I opened a feature request ticket:
>>> http://bugs.python.org/issue6783
>>>
>>> Some of the questions that remain:
>>> 1. Whether this should be a method for int or a regular function in a
>>> standard library module like math.
>>> 2. What should the method/function be called? (base_convert, radix, etc)
>>>
>>> What do you guys think?
>>
>> This has been coming up for years and always gets bogged down in a
>> spelling argument (a method on int, a function in the math module and an
>> update to the str.format mini language would be the current contenders).
>>
>> However, most of the actual real use cases for bases between 2 and 36
>> were dealt with by the addition of binary and octal output to string
>> formatting so the impetus to do anything about it is now a lot lower.
>>
>> As far as bases between 37 and 62 go, that would involve first getting
>> agreement on extending int() to handle those bases by allowing case
>> sensitive digit parsing. Presumably that would use string lexical
>> ordering so that int('a', 37) > int('A', 37) and int('b', 37) would
>> raise an exception.
>>
>> That would only be intuitive to someone that knows how ASCII based
>> alphanumeric ordering works though.

ASCII? Surely it should be Unicode! :-)

> Or it could be handled via a translation table (needed both ways of 
> course) mapping n indexes to n characters (with n the base you're 
> working with), defaulting to something sane.
> 
The default could cover only bases 2 to 36. Any base > 36 would require
a user-supplied translation table.

> Though I'm not sure this is of much interest really: even Erlang (which 
> provides pretty good base conversion tools: it supports literal integers 
> of any base between 2 and 36) doesn't natively support bases beyond 36. 
> A library would probably be better for those more conflictual (or less 
> intuitive) ranges.
> 
It could permit a dict as the translation table when 'decoding' so that
both 'A' and 'a' could be mapped to 10, if necessary.