From janssen at parc.com  Tue Mar  2 19:35:23 2010
From: janssen at parc.com (Bill Janssen)
Date: Tue, 2 Mar 2010 10:35:23 PST
Subject: [Python-ideas] the Quantity pattern
Message-ID: <57533.1267554923@parc.com>

I was looking at Martin Fowler's Quantity pattern earlier.

http://martinfowler.com/ap2/quantity.html

I remember writing this up as an idea for Fortran back in the early
80's, only to find a CACM paper from 1978 exploring the idea:
"Incorporation of Units into Programming Languages", Karr & Loveman, May
1978.

But it would still be a cool idea for Python.  Perhaps it's already
there and I haven't noticed?

Bill


From bwinton at latte.ca  Tue Mar  2 20:14:41 2010
From: bwinton at latte.ca (Blake Winton)
Date: Tue, 02 Mar 2010 14:14:41 -0500
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <57533.1267554923@parc.com>
References: <57533.1267554923@parc.com>
Message-ID: <4B8D63A1.2070409@latte.ca>

On 10-03-02 13:35 , Bill Janssen wrote:
> I was looking at Martin Fowler's Quantity pattern earlier.
>
> http://martinfowler.com/ap2/quantity.html
>
> I remember writing this up as an idea for Fortran back in the early
> 80's, only to find a CACM paper from 1978 exploring the idea:
> "Incorporation of Units into Programming Languages", Karr&  Loveman, May
> 1978.
>
> But it would still be a cool idea for Python.  Perhaps it's already
> there and I haven't noticed?

Were you thinking of something like 
http://www.arandonohue.com/hg/hgwebdir.cgi/units/file/bde33c01abd3/units/__init__.py#l1 
?

Later,
Blake.


From robert.kern at gmail.com  Tue Mar  2 20:17:06 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 02 Mar 2010 13:17:06 -0600
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <57533.1267554923@parc.com>
References: <57533.1267554923@parc.com>
Message-ID: <hmjo7k$ejp$1@dough.gmane.org>

On 2010-03-02 12:35 PM, Bill Janssen wrote:
> I was looking at Martin Fowler's Quantity pattern earlier.
>
> http://martinfowler.com/ap2/quantity.html
>
> I remember writing this up as an idea for Fortran back in the early
> 80's, only to find a CACM paper from 1978 exploring the idea:
> "Incorporation of Units into Programming Languages", Karr&  Loveman, May
> 1978.
>
> But it would still be a cool idea for Python.  Perhaps it's already
> there and I haven't noticed?

Tons of implementations (in no particular order):

http://pypi.python.org/pypi/quantities/
http://pypi.python.org/pypi/Unum/
http://pypi.python.org/pypi/magnitude/
http://pypi.python.org/pypi/units/
http://pypi.python.org/pypi/ScientificPython/
http://pypi.python.org/pypi/SciMath/

And quite a few more that are part of other packages or otherwise not on PyPI. 
It's ridiculously easy to write something that what people think are the common 
cases and so everyone does. It's a lot harder to write something that robustly 
handles what are actually common cases (absolute temperature scales, logarithmic 
scales, etc.).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From george.sakkis at gmail.com  Tue Mar  2 20:37:29 2010
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 2 Mar 2010 20:37:29 +0100
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <hmjo7k$ejp$1@dough.gmane.org>
References: <57533.1267554923@parc.com> <hmjo7k$ejp$1@dough.gmane.org>
Message-ID: <91ad5bf81003021137l2cbad439w21cd0f7e09a1a91d@mail.gmail.com>

On Tue, Mar 2, 2010 at 8:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2010-03-02 12:35 PM, Bill Janssen wrote:
>>
>> I was looking at Martin Fowler's Quantity pattern earlier.
>>
>> http://martinfowler.com/ap2/quantity.html
>>
>> I remember writing this up as an idea for Fortran back in the early
>> 80's, only to find a CACM paper from 1978 exploring the idea:
>> "Incorporation of Units into Programming Languages", Karr& ?Loveman, May
>> 1978.
>>
>> But it would still be a cool idea for Python. ?Perhaps it's already
>> there and I haven't noticed?
>
> Tons of implementations (in no particular order):
>
> http://pypi.python.org/pypi/quantities/
> http://pypi.python.org/pypi/Unum/
> http://pypi.python.org/pypi/magnitude/
> http://pypi.python.org/pypi/units/
> http://pypi.python.org/pypi/ScientificPython/
> http://pypi.python.org/pypi/SciMath/
>
> And quite a few more that are part of other packages or otherwise not on
> PyPI. It's ridiculously easy to write something that what people think are
> the common cases and so everyone does. It's a lot harder to write something
> that robustly handles what are actually common cases (absolute temperature
> scales, logarithmic scales, etc.).

One more: http://pypi.python.org/pypi/piquant/

I can't comment on its robustness and performance but as far as
readability goes, Unum seems the best of the bunch.

George


From anfedorov at gmail.com  Tue Mar  2 21:39:57 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Tue, 2 Mar 2010 15:39:57 -0500
Subject: [Python-ideas] Matching multiple regex patterns simultaneously
Message-ID: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>

So a couple of libraries (Django being the most popular that comes to mind)
try to match a string against several regex expressions. I'm wondering if
there exists a library to "merge" multiple compiled regex expressions into a
single lookup. This could be exposed in a interface like:

http://gist.github.com/319905


So for an example:

rd = ReDict()

rd['^foo$'] = 1
rd['^bar*$'] = 2
rd['^bar$'] = 3

assert rd['foo'] == [1]
assert rd['barrrr'] == [2]
assert rd['bar'] == [2,3]

The naive implementation I link is obviously inefficient. What would be the
easiest way to go about compiling a set of regex-es together, so that they
can be matched against a string at the same time? Are there any standard
libraries that do this I'm not aware of?

Cheers,
Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100302/f8016a57/attachment.html>

From mal at egenix.com  Tue Mar  2 23:09:34 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 02 Mar 2010 23:09:34 +0100
Subject: [Python-ideas] Matching multiple regex patterns simultaneously
In-Reply-To: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
References: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
Message-ID: <4B8D8C9E.5060201@egenix.com>

Andrey Fedorov wrote:
> So a couple of libraries (Django being the most popular that comes to mind)
> try to match a string against several regex expressions. I'm wondering if
> there exists a library to "merge" multiple compiled regex expressions into a
> single lookup. This could be exposed in a interface like:
> 
> http://gist.github.com/319905
> 
> 
> So for an example:
> 
> rd = ReDict()
> 
> rd['^foo$'] = 1
> rd['^bar*$'] = 2
> rd['^bar$'] = 3
> 
> assert rd['foo'] == [1]
> assert rd['barrrr'] == [2]
> assert rd['bar'] == [2,3]
> 
> The naive implementation I link is obviously inefficient. What would be the
> easiest way to go about compiling a set of regex-es together, so that they
> can be matched against a string at the same time? Are there any standard
> libraries that do this I'm not aware of?

This is a rather difficult problem to solve in general.

If you only need to search for a finite set of words, there are few
good algorithms for this:

http://en.wikipedia.org/wiki/Aho-Corasick_algorithm
 * used in Unix fgrep
 * Python implementation:
   http://hkn.eecs.berkeley.edu/~dyoo/python/ahocorasick/

http://en.wikipedia.org/wiki/Rabin-Karp_string_search_algorithm#Rabin.E2.80.93Karp_and_multiple_pattern_search
 * uses hashing

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 02 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From grosser.meister.morti at gmx.net  Tue Mar  2 23:32:42 2010
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Tue, 02 Mar 2010 23:32:42 +0100
Subject: [Python-ideas] Matching multiple regex patterns simultaneously
In-Reply-To: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
References: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
Message-ID: <4B8D920A.1010309@gmx.net>

On 03/02/2010 09:39 PM, Andrey Fedorov wrote:
> So a couple of libraries (Django being the most popular that comes to
> mind) try to match a string against several regex expressions. I'm
> wondering if there exists a library to "merge" multiple compiled regex
> expressions into a single lookup. This could be exposed in a interface like:
>
>     http://gist.github.com/319905
>
>
> So for an example:
>
> rd = ReDict()
>
> rd['^foo$'] = 1
> rd['^bar*$'] = 2
> rd['^bar$'] = 3
>
> assert rd['foo'] == [1]
> assert rd['barrrr'] == [2]
> assert rd['bar'] == [2,3]
>
> The naive implementation I link is obviously inefficient. What would be
> the easiest way to go about compiling a set of regex-es together, so
> that they can be matched against a string at the same time? Are there
> any standard libraries that do this I'm not aware of?
>
> Cheers,
> Andrey
>

You can do something like this:
r=re.compile('(?P<a>^foo$)|(?P<b>(?P<c>^bar)r*$)')
 >>> r.match('barrrr').groupdict()
{'a': None, 'c': 'bar', 'b': 'barrrr'}
 >>> r.match('bar').groupdict()
{'a': None, 'c': 'bar', 'b': 'bar'}
 >>> r.match('foo').groups()
('foo', None, None)

Ok, it's not 100% the same (it does not match 'ba'), but I think this should cover most cases where 
you want something like this. Hmm, well. You should resolve it to a form where there are no 
overlappings in the subexpressions:
(?P<a>^foo$)|(?P<b>^ba$)|(?P<c>^bar$)|(?P<d>^bar+$)

	-panzi


From mwm-keyword-python.b4bdba at mired.org  Wed Mar  3 01:17:43 2010
From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer)
Date: Tue, 2 Mar 2010 19:17:43 -0500
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <91ad5bf81003021137l2cbad439w21cd0f7e09a1a91d@mail.gmail.com>
References: <57533.1267554923@parc.com> <hmjo7k$ejp$1@dough.gmane.org>
	<91ad5bf81003021137l2cbad439w21cd0f7e09a1a91d@mail.gmail.com>
Message-ID: <20100302191743.7d7b6f77@bhuda.mired.org>

On Tue, 2 Mar 2010 20:37:29 +0100
George Sakkis <george.sakkis at gmail.com> wrote:

> On Tue, Mar 2, 2010 at 8:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
> > On 2010-03-02 12:35 PM, Bill Janssen wrote:
> >>
> >> I was looking at Martin Fowler's Quantity pattern earlier.
> >>
> >> http://martinfowler.com/ap2/quantity.html
> >>
> >> I remember writing this up as an idea for Fortran back in the early
> >> 80's, only to find a CACM paper from 1978 exploring the idea:
> >> "Incorporation of Units into Programming Languages", Karr& ?Loveman, May
> >> 1978.
> >>
> >> But it would still be a cool idea for Python. ?Perhaps it's already
> >> there and I haven't noticed?
> >
> > Tons of implementations (in no particular order):
> >
> > http://pypi.python.org/pypi/quantities/
> > http://pypi.python.org/pypi/Unum/
> > http://pypi.python.org/pypi/magnitude/
> > http://pypi.python.org/pypi/units/
> > http://pypi.python.org/pypi/ScientificPython/
> > http://pypi.python.org/pypi/SciMath/
> >
> > And quite a few more that are part of other packages or otherwise not on
> > PyPI. It's ridiculously easy to write something that what people think are
> > the common cases and so everyone does. It's a lot harder to write something
> > that robustly handles what are actually common cases (absolute temperature
> > scales, logarithmic scales, etc.).
> 
> One more: http://pypi.python.org/pypi/piquant/
> 
> I can't comment on its robustness and performance but as far as
> readability goes, Unum seems the best of the bunch.

Hmm. How about accessing the Frink
(http://futureboy.homeip.net/frinkdocs/) types from Jython?

	 <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org


From robert.kern at gmail.com  Wed Mar  3 01:42:26 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 02 Mar 2010 18:42:26 -0600
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <20100302191743.7d7b6f77@bhuda.mired.org>
References: <57533.1267554923@parc.com>
	<hmjo7k$ejp$1@dough.gmane.org>	<91ad5bf81003021137l2cbad439w21cd0f7e09a1a91d@mail.gmail.com>
	<20100302191743.7d7b6f77@bhuda.mired.org>
Message-ID: <hmkb9j$mik$1@dough.gmane.org>

On 2010-03-02 18:17 PM, Mike Meyer wrote:
> On Tue, 2 Mar 2010 20:37:29 +0100
> George Sakkis<george.sakkis at gmail.com>  wrote:
>
>> On Tue, Mar 2, 2010 at 8:17 PM, Robert Kern<robert.kern at gmail.com>  wrote:
>>> On 2010-03-02 12:35 PM, Bill Janssen wrote:
>>>>
>>>> I was looking at Martin Fowler's Quantity pattern earlier.
>>>>
>>>> http://martinfowler.com/ap2/quantity.html
>>>>
>>>> I remember writing this up as an idea for Fortran back in the early
>>>> 80's, only to find a CACM paper from 1978 exploring the idea:
>>>> "Incorporation of Units into Programming Languages", Karr&    Loveman, May
>>>> 1978.
>>>>
>>>> But it would still be a cool idea for Python.  Perhaps it's already
>>>> there and I haven't noticed?
>>>
>>> Tons of implementations (in no particular order):
>>>
>>> http://pypi.python.org/pypi/quantities/
>>> http://pypi.python.org/pypi/Unum/
>>> http://pypi.python.org/pypi/magnitude/
>>> http://pypi.python.org/pypi/units/
>>> http://pypi.python.org/pypi/ScientificPython/
>>> http://pypi.python.org/pypi/SciMath/
>>>
>>> And quite a few more that are part of other packages or otherwise not on
>>> PyPI. It's ridiculously easy to write something that what people think are
>>> the common cases and so everyone does. It's a lot harder to write something
>>> that robustly handles what are actually common cases (absolute temperature
>>> scales, logarithmic scales, etc.).
>>
>> One more: http://pypi.python.org/pypi/piquant/
>>
>> I can't comment on its robustness and performance but as far as
>> readability goes, Unum seems the best of the bunch.
>
> Hmm. How about accessing the Frink
> (http://futureboy.homeip.net/frinkdocs/) types from Jython?

You might be able to use it to do the unit conversion calculations, but the Java 
interface to Frink does not expose any types following the Quantity pattern. It 
just provides a way to evaluate strings in the Frink interpreter:

   http://futureboy.homeip.net/frinkdocs/integrate/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From dsdale24 at gmail.com  Wed Mar  3 03:02:31 2010
From: dsdale24 at gmail.com (Darren Dale)
Date: Tue, 2 Mar 2010 21:02:31 -0500
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <91ad5bf81003021137l2cbad439w21cd0f7e09a1a91d@mail.gmail.com>
References: <57533.1267554923@parc.com> <hmjo7k$ejp$1@dough.gmane.org>
	<91ad5bf81003021137l2cbad439w21cd0f7e09a1a91d@mail.gmail.com>
Message-ID: <a08e5f81003021802n64b1deb7w4460c5a98bcdbac9@mail.gmail.com>

On Tue, Mar 2, 2010 at 2:37 PM, George Sakkis <george.sakkis at gmail.com> wrote:
> On Tue, Mar 2, 2010 at 8:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
>> On 2010-03-02 12:35 PM, Bill Janssen wrote:
>>>
>>> I was looking at Martin Fowler's Quantity pattern earlier.
>>>
>>> http://martinfowler.com/ap2/quantity.html
>>>
>>> I remember writing this up as an idea for Fortran back in the early
>>> 80's, only to find a CACM paper from 1978 exploring the idea:
>>> "Incorporation of Units into Programming Languages", Karr& ?Loveman, May
>>> 1978.
>>>
>>> But it would still be a cool idea for Python. ?Perhaps it's already
>>> there and I haven't noticed?
>>
>> Tons of implementations (in no particular order):
>>
>> http://pypi.python.org/pypi/quantities/
>> http://pypi.python.org/pypi/Unum/
>> http://pypi.python.org/pypi/magnitude/
>> http://pypi.python.org/pypi/units/
>> http://pypi.python.org/pypi/ScientificPython/
>> http://pypi.python.org/pypi/SciMath/
>>
>> And quite a few more that are part of other packages or otherwise not on
>> PyPI. It's ridiculously easy to write something that what people think are
>> the common cases and so everyone does. It's a lot harder to write something
>> that robustly handles what are actually common cases (absolute temperature
>> scales, logarithmic scales, etc.).
>
> One more: http://pypi.python.org/pypi/piquant/
>
> I can't comment on its robustness and performance but as far as
> readability goes, Unum seems the best of the bunch.

I am the developer of the Quantities package. The tutorial at
http://packages.python.org/quantities/user/tutorial.html recommends
importing the units and constants into a namespace, but aside from
that, the syntax seems very similar to Unum. However, quantities
depends on numpy.

Darren


From dsdale24 at gmail.com  Wed Mar  3 03:14:57 2010
From: dsdale24 at gmail.com (Darren Dale)
Date: Tue, 2 Mar 2010 21:14:57 -0500
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <hmjo7k$ejp$1@dough.gmane.org>
References: <57533.1267554923@parc.com> <hmjo7k$ejp$1@dough.gmane.org>
Message-ID: <a08e5f81003021814k533e8be6l552aafbd3a1d6ec9@mail.gmail.com>

On Tue, Mar 2, 2010 at 2:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2010-03-02 12:35 PM, Bill Janssen wrote:
>>
>> I was looking at Martin Fowler's Quantity pattern earlier.
>>
>> http://martinfowler.com/ap2/quantity.html
>>
>> I remember writing this up as an idea for Fortran back in the early
>> 80's, only to find a CACM paper from 1978 exploring the idea:
>> "Incorporation of Units into Programming Languages", Karr& ?Loveman, May
>> 1978.
>>
>> But it would still be a cool idea for Python. ?Perhaps it's already
>> there and I haven't noticed?
>
> Tons of implementations (in no particular order):
>
> http://pypi.python.org/pypi/quantities/
> http://pypi.python.org/pypi/Unum/
> http://pypi.python.org/pypi/magnitude/
> http://pypi.python.org/pypi/units/
> http://pypi.python.org/pypi/ScientificPython/
> http://pypi.python.org/pypi/SciMath/
>
> And quite a few more that are part of other packages or otherwise not on
> PyPI. It's ridiculously easy to write something that what people think are
> the common cases and so everyone does. It's a lot harder to write something
> that robustly handles what are actually common cases (absolute temperature
> scales, logarithmic scales, etc.).

I prefer to think of this as two separate issues. One issue is a
Quantity pattern for dealing with values that have magnitude and
dimensionality, and the other is coordinate systems (requiring a point
of reference, like temperature scales).

Darren


From robert.kern at gmail.com  Wed Mar  3 04:35:00 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 02 Mar 2010 21:35:00 -0600
Subject: [Python-ideas] the Quantity pattern
In-Reply-To: <a08e5f81003021814k533e8be6l552aafbd3a1d6ec9@mail.gmail.com>
References: <57533.1267554923@parc.com> <hmjo7k$ejp$1@dough.gmane.org>
	<a08e5f81003021814k533e8be6l552aafbd3a1d6ec9@mail.gmail.com>
Message-ID: <hmkldm$etv$1@dough.gmane.org>

On 2010-03-02 20:14 , Darren Dale wrote:
> On Tue, Mar 2, 2010 at 2:17 PM, Robert Kern<robert.kern at gmail.com>  wrote:
>> On 2010-03-02 12:35 PM, Bill Janssen wrote:
>>>
>>> I was looking at Martin Fowler's Quantity pattern earlier.
>>>
>>> http://martinfowler.com/ap2/quantity.html
>>>
>>> I remember writing this up as an idea for Fortran back in the early
>>> 80's, only to find a CACM paper from 1978 exploring the idea:
>>> "Incorporation of Units into Programming Languages", Karr&    Loveman, May
>>> 1978.
>>>
>>> But it would still be a cool idea for Python.  Perhaps it's already
>>> there and I haven't noticed?
>>
>> Tons of implementations (in no particular order):
>>
>> http://pypi.python.org/pypi/quantities/
>> http://pypi.python.org/pypi/Unum/
>> http://pypi.python.org/pypi/magnitude/
>> http://pypi.python.org/pypi/units/
>> http://pypi.python.org/pypi/ScientificPython/
>> http://pypi.python.org/pypi/SciMath/
>>
>> And quite a few more that are part of other packages or otherwise not on
>> PyPI. It's ridiculously easy to write something that what people think are
>> the common cases and so everyone does. It's a lot harder to write something
>> that robustly handles what are actually common cases (absolute temperature
>> scales, logarithmic scales, etc.).
>
> I prefer to think of this as two separate issues. One issue is a
> Quantity pattern for dealing with values that have magnitude and
> dimensionality, and the other is coordinate systems (requiring a point
> of reference, like temperature scales).

Theoretically and implementation-wise, absolutely. However, users want to 
convert Fahrenheit to Celsius with the same tool they use to convert meters to 
feet. To them, it's the same problem.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From stefan_ml at behnel.de  Wed Mar  3 09:37:27 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 03 Mar 2010 09:37:27 +0100
Subject: [Python-ideas] Matching multiple regex patterns simultaneously
In-Reply-To: <4B8D8C9E.5060201@egenix.com>
References: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
	<4B8D8C9E.5060201@egenix.com>
Message-ID: <hml747$pkp$1@dough.gmane.org>

M.-A. Lemburg, 02.03.2010 23:09:
> If you only need to search for a finite set of words, there are few
> good algorithms for this:
>
> http://en.wikipedia.org/wiki/Aho-Corasick_algorithm
>   * used in Unix fgrep
>   * Python implementation:
>     http://hkn.eecs.berkeley.edu/~dyoo/python/ahocorasick/
>
> http://en.wikipedia.org/wiki/Rabin-Karp_string_search_algorithm#Rabin.E2.80.93Karp_and_multiple_pattern_search
>   * uses hashing

... and acora:

http://pypi.python.org/pypi/acora

Stefan



From mikegraham at gmail.com  Wed Mar  3 20:48:18 2010
From: mikegraham at gmail.com (Mike Graham)
Date: Wed, 3 Mar 2010 13:48:18 -0600
Subject: [Python-ideas] Function caller in operator module
Message-ID: <3f588a651003031148gc35ca37o7bb609c02f2e61fa@mail.gmail.com>

Most operations that are available using operators in Python are
provided by the operator module, but calling functions is noticeably
absent is calling functions. A user can slightly abuse
operator.methodcaller('__call__', ...) to perform this operation, but
that is far from ideal.

Should operator grow a new function caller, such that
operator.caller(*args, **kwargs)(f) returns f(*args, **kwargs)?

I have never personally needed this exact operation; it would be
somewhat odd but not altogether unthinkable to sort a list of
callables by their return values. The closes thing to this I have seen
in the wild is map(apply, fs), which is limited to the no-arguments
case. It is also possibly uglier than [f() for f in fs]. In any event,
it's not even an option in Python 3.x.


From tjreedy at udel.edu  Wed Mar  3 22:21:16 2010
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 03 Mar 2010 16:21:16 -0500
Subject: [Python-ideas] Function caller in operator module
In-Reply-To: <3f588a651003031148gc35ca37o7bb609c02f2e61fa@mail.gmail.com>
References: <3f588a651003031148gc35ca37o7bb609c02f2e61fa@mail.gmail.com>
Message-ID: <hmmjsb$5og$2@dough.gmane.org>

On 3/3/2010 2:48 PM, Mike Graham wrote:
> Most operations that are available using operators in Python are
> provided by the operator module, but calling functions is noticeably
> absent is calling functions. A user can slightly abuse
> operator.methodcaller('__call__', ...) to perform this operation, but
> that is far from ideal.
>
> Should operator grow a new function caller, such that
> operator.caller(*args, **kwargs)(f) returns f(*args, **kwargs)?

Use case?

> I have never personally needed this exact operation; it would be
> somewhat odd but not altogether unthinkable to sort a list of
> callables by their return values. The closes thing to this I have seen
> in the wild is map(apply, fs), which is limited to the no-arguments
> case. It is also possibly uglier than [f() for f in fs]. In any event,
> it's not even an option in Python 3.x.

And not needed, as you show.

tjr





From mal at egenix.com  Thu Mar  4 01:21:07 2010
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 04 Mar 2010 01:21:07 +0100
Subject: [Python-ideas] Matching multiple regex patterns simultaneously
In-Reply-To: <hml747$pkp$1@dough.gmane.org>
References: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>	<4B8D8C9E.5060201@egenix.com>
	<hml747$pkp$1@dough.gmane.org>
Message-ID: <4B8EFCF3.80109@egenix.com>

Stefan Behnel wrote:
> M.-A. Lemburg, 02.03.2010 23:09:
>> If you only need to search for a finite set of words, there are few
>> good algorithms for this:
>>
>> http://en.wikipedia.org/wiki/Aho-Corasick_algorithm
>>   * used in Unix fgrep
>>   * Python implementation:
>>     http://hkn.eecs.berkeley.edu/~dyoo/python/ahocorasick/
>>
>> http://en.wikipedia.org/wiki/Rabin-Karp_string_search_algorithm#Rabin.E2.80.93Karp_and_multiple_pattern_search
>>
>>   * uses hashing
> 
> ... and acora:
> 
> http://pypi.python.org/pypi/acora

Nice :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 04 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/


From denis.spir at gmail.com  Thu Mar  4 11:35:53 2010
From: denis.spir at gmail.com (spir)
Date: Thu, 4 Mar 2010 11:35:53 +0100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
Message-ID: <20100304113553.514db067@o>

Hello,

(1) I do not understand an iterable type's __iter__() method to be compulsary. Actually, each time I have defined one, I had to write:
    def __iter__(self):
        return self
So, I guess that if python does not find __iter__(), but the object defines next(), then by default the said object could be used as its own iterator. This is what I understand by "iterable" and next() is the required method for it.
Or even better: only if the object does not define next(), then python falls back to looking for __iter__(). Is there any obstacle for this I cannot see?
Side-question: In which cases is it necessary to define the iterator as a separate object?

(2) But: for any reason next() is not spelled as a "magic" method. If this method becomes the distinctive method of iterables, then it should be called __next__() for consistency.
Side-question: Why is it called next(), as it is a magic method for iterators already?

(3) What I miss actually for iterables (which are their own iterator) is a kind of __reset__(). In some cases, it is only needed to allow a new iteration from start. But it may even be needed to set some startup data the first time. __reset__() would thus be called once before the first call to next(). (Sure, "reset" may not be the best term. Maybe "begin" or "startup"? The sense is: "Prepare to yield the first item!")
In absence of such a startup mechanism, I end up using __call__ instead: Example:

class Powers(object):
    def __init__(self, exponent):
        self.exponent = exponent
    def next(self):
        n = self.n + 1
        if (self.max is not None) and (n > self.max):
            raise StopIteration
        self.n = n
        return n*n
    def __call__(self, min=1,max=None):
        self.n = min-1
        self.max = max
        return self	#.__iter__()
    def __iter__(self):
        return self

tripleCubes = []
cubes = Powers(3)
for sq in cubes(7,17):
    if sq%3 == 0:
        tripleCubes.append(sq)
print tripleCubes    # ==> [81, 144, 225]

__iter__() could be used directly if it would allow "free" args in addition to self (in this case: def __iter__(self, min=0,max=None). This beeing impossible, an aditional method seems to be needed.

To sum up, I would enjoy beeing able to write Powers using a scheme like:
class Powers(object):
    def __init__(self, exponent):
        self.exponent = exponent
    def __reset__(self, min=1,max=None):
        self.n = min-1
        self.max = max
    def __next__(self):
        n = self.n + 1
        if (self.max is not None) and (n > self.max):
            raise StopIteration
        self.n = n
        return n*n

This matches the overall idea of iterable for me.

Denis
-- 
________________________________

la vita e estrany

spir.wikidot.com



From masklinn at masklinn.net  Thu Mar  4 12:07:48 2010
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 4 Mar 2010 12:07:48 +0100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <20100304113553.514db067@o>
References: <20100304113553.514db067@o>
Message-ID: <B98A2072-FA8D-44B5-837C-B337CDB7C170@masklinn.net>

On 4 Mar 2010, at 11:35 , spir wrote:
> 
> (3) What I miss actually for iterables (which are their own iterator) is a kind of __reset__(). In some cases, it is only needed to allow a new iteration from start. But it may even be needed to set some startup data the first time. __reset__() would thus be called once before the first call to next(). (Sure, "reset" may not be the best term. Maybe "begin" or "startup"? The sense is: "Prepare to yield the first item!")

The use case you propose (and demonstrate in your example) shows that you're creating an iterating view over your container. Which doesn't seem the role of __iter__ at all as far as I understand, so you should indeed use either __call__ or a separate method call. Slice, for instance. In fact, in your case I think supporting slicing/islicing would make far more sense than the solution you've elected):

    tripleCubes = []
    cubes = Powers(3)
    for sq in cubes[6, 17]:
        if sq%3 == 0:
            tripleCubes.append(sq)

You could also compose your iterable with existing tools such as those in ``itertools``, or create your own composable iterable transformers/manipulators (I strongly recommend David M. Beazley's presentations on generators for that [1][2].

With those, removing __call__ from your Power[3] class and setting `self.n = 0` (and `self.max = None`) in the constructor:

    cubes = Powers(3)
    tripleCubes = filter(lambda sq: sq%3 == 0, (islice(cubes, 6, 17)))

Or (to avoid the pretty ugly lambda and use a listcomp):

    cubes = Powers(3)
    tripleCubes = [sq for sq in islice(cubes, 6, 16)
                      if sq%3 == 0]

[1] Generator Tricks for Systems Programmers http://www.dabeaz.com/generators/
[2] A Curious Course on Coroutines and Concurrency http://www.dabeaz.com/coroutines/
[3] I hope and believe you wouldn't actually write such code outside of an example as there are much better ways to achieve the same thing in Python

From pyideas at rebertia.com  Thu Mar  4 12:45:11 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Thu, 4 Mar 2010 03:45:11 -0800
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <20100304113553.514db067@o>
References: <20100304113553.514db067@o>
Message-ID: <50697b2c1003040345o24713191w54902939c326ab99@mail.gmail.com>

On Thu, Mar 4, 2010 at 2:35 AM, spir <denis.spir at gmail.com> wrote:
> Hello,
>
> (1) I do not understand an iterable type's __iter__() method to be compulsary. Actually, each time I have defined one, I had to write:
> ? ?def __iter__(self):
> ? ? ? ?return self
> So, I guess that if python does not find __iter__(), but the object defines next(), then by default the said object could be used as its own iterator. This is what I understand by "iterable" and next() is the required method for it.
> Or even better: only if the object does not define next(), then python falls back to looking for __iter__(). Is there any obstacle for this I cannot see?

Not that I can think of; Python just happened to make a different
design decision than you, one that simplifies (and likely speeds up)
the interpreter at the minor cost of having to write a trivial "return
self" __iter__() in some cases: the interpreter can just blindly call
__iter__() as opposed to (as you suggest) doing more sophisticated
checking for a next() method and only then falling back to __iter__().

> Side-question: In which cases is it necessary to define the iterator as a separate object?

Whenever you want to use multiple iterators over the same object
simultaneously (you can usually equally use a generator instead of an
object, but the iterator is separate from the iterate-ee in either
case). For example, if lists were their own iterators, the following
code:

for item1 in some_list:
    for item2 in some_list:
        print item1, item2

rather than outputting the cross-product of the items in the list,
would presumably instead output the first element paired with every
other element in the list, which is not what was intended.

> (2) But: for any reason next() is not spelled as a "magic" method. If this method becomes the distinctive method of iterables, then it should be called __next__() for consistency.

Guido's time machine strikes again! This is fixed in Python 3.x:
http://www.python.org/dev/peps/pep-3114/

> (3) What I miss actually for iterables (which are their own iterator) is a kind of __reset__(). In some cases, it is only needed to allow a new iteration from start. But it may even be needed to set some startup data the first time. __reset__() would thus be called once before the first call to next().

(a) __reset__() shouldn't be part of the iterator protocol since it's
not applicable for all iterators, only some.
(b) You can just write a generator and put the setup code before the
initial "yield" to much the same effect.

Cheers,
Chris
--
Maven & would-be designer of languages
http://blog.rebertia.com


From denis.spir at gmail.com  Thu Mar  4 12:53:27 2010
From: denis.spir at gmail.com (spir)
Date: Thu, 4 Mar 2010 12:53:27 +0100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <B98A2072-FA8D-44B5-837C-B337CDB7C170@masklinn.net>
References: <20100304113553.514db067@o>
	<B98A2072-FA8D-44B5-837C-B337CDB7C170@masklinn.net>
Message-ID: <20100304125327.436b1801@o>

On Thu, 4 Mar 2010 12:07:48 +0100
Masklinn <masklinn at masklinn.net> wrote:

Thanks for your reply: I find it helpful; will have a look at the pointed references in a short while.

> In fact, in your case I think supporting slicing/islicing would make far more sense than the solution you've elected)

Actually my Powers type should be "sliceable".

>     tripleCubes = [sq for sq in islice(cubes, 6, 16)
>                       if sq%3 == 0]

Right, but this pattern only replaces cubes(6, 16) by islice(cubes, 6, 16):
    tripleCubes = [sq for sq in cubes(6, 16) if sq%3 == 0]
Or do I overlook a relevant point? I guess the proper (and pythonic?) solution would be to implement __getitem__ so as to be able to write:
    tripleCubes = [sq for sq in cubes[6:17] if sq%3 == 0]
Does the following match your idea?
    def __getitem__(self, ranj):
        self.n , self.max = ranj.start-1 , ranj.stop-1
        return self

Denis
-- 
________________________________

la vita e estrany

spir.wikidot.com



From masklinn at masklinn.net  Thu Mar  4 13:41:14 2010
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 4 Mar 2010 13:41:14 +0100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <20100304125327.436b1801@o>
References: <20100304113553.514db067@o>
	<B98A2072-FA8D-44B5-837C-B337CDB7C170@masklinn.net>
	<20100304125327.436b1801@o>
Message-ID: <FA2B87EE-7288-4474-AE88-78AAA2843A0D@masklinn.net>

On 4 Mar 2010, at 12:53 , spir wrote:
> 
> Right, but this pattern only replaces cubes(6, 16) by islice(cubes, 6, 16)
Yes, the point is that you don't *need* cubes to be callable to do what you want. Because Python already provides the tools to do it in its stdlib. So you have no reason to concern yourself with that.

> Does the following match your idea?
>    def __getitem__(self, ranj):
>        self.n , self.max = ranj.start-1 , ranj.stop-1
>        return self
I'd return a new item with the relevant attributes set, not an modified set (generally, I'm not fond of mutable object but that might just be me)



From merwok at netwok.org  Thu Mar  4 13:37:56 2010
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 04 Mar 2010 13:37:56 +0100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <20100304113553.514db067@o>
References: <20100304113553.514db067@o>
Message-ID: <4B8FA9A4.2090907@netwok.org>

Hello list

Some complementary information to Chris Rebert?s message.

> (1) I do not understand an iterable type's __iter__() method to be
 > compulsary. [...]
The iterable and the iterator protocols are two different things. Every 
iterator is iterable, but not every iterable object is an iterator (see 
<http://docs.python.org/library/collections>). There are situations 
where it makes sense to use a different class as iterator (can?t find 
examples right now, hope other people will chime in), and a lot of 
situations where it?s fine to implement both protocols on the same class.

> Actually, each time I have defined one, I had to write:
>     def __iter__(self):
>         return self
Instead of returning self in __iter__ and then defining a function in 
next, i.e. returning values and raising StopIteration, you can define 
__iter__ as a generator (i.e. yield values instead of returning them) 
and not need to write next. Helpful documentation here:
<http://docs.python.org/reference/datamodel#object.__iter__> and
<http://docs.python.org/library/stdtypes#typeiter>


> (3) What I miss actually for iterables (which are their own iterator)
 > is a kind of __reset__(). In some cases, it is only needed to allow a
 > new iteration from start.[...]
Didn?t really understand your use case here, but perhaps the extension 
of the yield statement done in version 2.5 can help you here:
<http://docs.python.org/reference/expressions#yieldexpr>
<http://www.python.org/dev/peps/pep-0342/>

Hope this helps.

Regards


From ncoghlan at gmail.com  Thu Mar  4 14:23:34 2010
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 04 Mar 2010 23:23:34 +1000
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <20100304113553.514db067@o>
References: <20100304113553.514db067@o>
Message-ID: <4B8FB456.5050309@gmail.com>

spir wrote:
> Hello,
> 
> (1) I do not understand an iterable type's __iter__() method to be
> compulsary. Actually, each time I have defined one, I had to write: 
> def __iter__(self): return self So, I guess that if python does not
> find __iter__(), but the object defines next(), then by default the
> said object could be used as its own iterator. This is what I
> understand by "iterable" and next() is the required method for it. Or
> even better: only if the object does not define next(), then python
> falls back to looking for __iter__(). Is there any obstacle for this
> I cannot see? Side-question: In which cases is it necessary to define
> the iterator as a separate object?

Almost all containers should use a separate object for their iterators.
Note that this is already the case for all of Python's standard
container types.

The reason relates to the __reset__ suggestion you describe later in
your message: How do I reset an iterator over a list? Easy, just call
iter() again - it will give me a fresh iterator that starts at the
beginning without affecting the list or my original iterator. By
producing a fresh object for each invocation of __iter__ the state of
the iterators is decoupled from the state of the underlying object which
is generally a good thing from a program design point of view.

(See Eric's suggestion regarding the use of generators as __iter__
methods to easily achieve this behaviour)

Objects with significant state that are also their own iterators are
actually quite rare. File objects certainly qualify (since they base
their iteration off the file object's file pointer), but I can't think
of any others off the top of my head.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------


From brett at python.org  Thu Mar  4 21:25:19 2010
From: brett at python.org (Brett Cannon)
Date: Thu, 4 Mar 2010 12:25:19 -0800
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <4B8FB456.5050309@gmail.com>
References: <20100304113553.514db067@o> <4B8FB456.5050309@gmail.com>
Message-ID: <bbaeab101003041225x72aed1f2v9f97b8788318400d@mail.gmail.com>

On Thu, Mar 4, 2010 at 05:23, Nick Coghlan <ncoghlan at gmail.com> wrote:

> spir wrote:
> > Hello,
> >
> > (1) I do not understand an iterable type's __iter__() method to be
> > compulsary. Actually, each time I have defined one, I had to write:
> > def __iter__(self): return self So, I guess that if python does not
> > find __iter__(), but the object defines next(), then by default the
> > said object could be used as its own iterator. This is what I
> > understand by "iterable" and next() is the required method for it. Or
> > even better: only if the object does not define next(), then python
> > falls back to looking for __iter__(). Is there any obstacle for this
> > I cannot see? Side-question: In which cases is it necessary to define
> > the iterator as a separate object?
>
> Almost all containers should use a separate object for their iterators.
> Note that this is already the case for all of Python's standard
> container types.
>

There is also the issue of backwards-compatibility when iterators were
introduced. Just because someone decided to have a method named next() when
iterators were introduced does not mean they intended for it to be viewed as
a sequence. Requiring an iterable to define __iter__() took care of the
ambiguity.

-Brett




>
> The reason relates to the __reset__ suggestion you describe later in
> your message: How do I reset an iterator over a list? Easy, just call
> iter() again - it will give me a fresh iterator that starts at the
> beginning without affecting the list or my original iterator. By
> producing a fresh object for each invocation of __iter__ the state of
> the iterators is decoupled from the state of the underlying object which
> is generally a good thing from a program design point of view.
>
> (See Eric's suggestion regarding the use of generators as __iter__
> methods to easily achieve this behaviour)
>
> Objects with significant state that are also their own iterators are
> actually quite rare. File objects certainly qualify (since they base
> their iteration off the file object's file pointer), but I can't think
> of any others off the top of my head.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100304/09a9012c/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Mar  4 23:48:06 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 05 Mar 2010 11:48:06 +1300
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <20100304113553.514db067@o>
References: <20100304113553.514db067@o>
Message-ID: <4B9038A6.2060909@canterbury.ac.nz>

spir wrote:

> (2) But: for any reason next() is not spelled as a "magic" method. If 
> this method becomes the distinctive method of iterables, then it 
> should be called __next__() for consistency.

I gather that it is indeed called __next__() in Py3, and there
is a new builtin function next() for invoking it.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Mar  5 00:07:55 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 05 Mar 2010 12:07:55 +1300
Subject: [Python-ideas] Matching multiple regex patterns simultaneously
In-Reply-To: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
References: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
Message-ID: <4B903D4B.1050209@canterbury.ac.nz>

Andrey Fedorov wrote:
> So a couple of libraries (Django being the most popular that comes to mind)
> try to match a string against several regex expressions. I'm wondering if
> there exists a library to "merge" multiple compiled regex expressions into a
> single lookup.

The only thing I'm aware of at the moment is my own
Plex library, but it's currently implemented in pure
Python, so it's not as efficient as the re module
with an extension like this would be.

I have a half-finished project on the back burner
to reimplement the core of Plex using Pyrex, but
it's been languishing for so long that the burner
has probably gone out by now. Maybe I should
relight it...

-- 
Greg


From g.brandl at gmx.net  Thu Mar  4 22:43:53 2010
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 04 Mar 2010 22:43:53 +0100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <20100304113553.514db067@o>
References: <20100304113553.514db067@o>
Message-ID: <hmp9hn$tr3$1@dough.gmane.org>

Am 04.03.2010 11:35, schrieb spir:
> Hello,
> 
> (1) I do not understand an iterable type's __iter__() method to be
> compulsary. Actually, each time I have defined one, I had to write: def
> __iter__(self): return self So, I guess that if python does not find
> __iter__(), but the object defines next(), then by default the said object
> could be used as its own iterator. This is what I understand by "iterable"
> and next() is the required method for it. Or even better: only if the object
> does not define next(), then python falls back to looking for __iter__(). Is
> there any obstacle for this I cannot see? Side-question: In which cases is it
> necessary to define the iterator as a separate object?

I would say that in most cases it makes sense to have the iterator be a separate
object.  However, when writing an __iter__() in Python, you almost always can
make it a generator, which already does this for you -- each call to __iter__()
will return a new generator.

> (2) But: for any reason next() is not spelled as a "magic" method. If this
> method becomes the distinctive method of iterables, then it should be called
> __next__() for consistency. Side-question: Why is it called next(), as it is
> a magic method for iterators already?

Because it is supposed to be called directly.  __iter__() isn't.  (This changes
with Python 3, where you have next() as a builtin.)  As such, this is the same
question as "why is it called readline(), not __readline__()."  readline(), just
like next(), is a method defined by a protocol (file vs iterator).

Georg



From solipsis at pitrou.net  Fri Mar  5 00:54:05 2010
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 4 Mar 2010 18:54:05 -0500
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
References: <20100304113553.514db067@o>
Message-ID: <20100304185405.4436a34c@msiwind>

Le Thu, 4 Mar 2010 11:35:53 +0100,
spir <denis.spir at gmail.com> a ?crit :
> 
> (1) I do not understand an iterable type's __iter__() method to be
> compulsary. Actually, each time I have defined one, I had to write:
> def __iter__(self): return self
> So, I guess that if python does not find __iter__(), but the object
> defines next(), then by default the said object could be used as its
> own iterator.

Explicit is better than implicit, though.
In many non-trivial cases, the iterator will have to be a separate
object anyway.
Also, please note you can implement __iter__ as a generator, which
makes things very easy for the simple cases:

>>> class C(object):
...   def __iter__(self):
...     yield 1
...     yield 2
... 
>>> c = C()
>>> it = iter(c)
>>> it
<generator object __iter__ at 0xb746c0f4>
>>> next(it)
1
>>> next(it)
2
>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> list(c)
[1, 2]

> (3) What I miss actually for iterables (which are their own iterator)
> is a kind of __reset__().

No, really, you don't want this, unless you like PHP. As others said,
calling iter() again is the well-defined generic way to "reset" your
iterable.
Then, particular cases can warrant specific APIs, such as file.seek().

Regards

Antoine.




From grosser.meister.morti at gmx.net  Fri Mar  5 05:01:27 2010
From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=)
Date: Fri, 05 Mar 2010 05:01:27 +0100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <hmp9hn$tr3$1@dough.gmane.org>
References: <20100304113553.514db067@o> <hmp9hn$tr3$1@dough.gmane.org>
Message-ID: <4B908217.9060603@gmx.net>

On 03/04/2010 10:43 PM, Georg Brandl wrote:
>> >  (2) But: for any reason next() is not spelled as a "magic" method. If this
>> >  method becomes the distinctive method of iterables, then it should be called
>> >  __next__() for consistency. Side-question: Why is it called next(), as it is
>> >  a magic method for iterators already?
> Because it is supposed to be called directly.  __iter__() isn't.  (This changes
> with Python 3, where you have next() as a builtin.)

And why is it made a builtin function? What was wrong with it being a normal method?

	-panzi


From andrew at bemusement.org  Fri Mar  5 05:17:59 2010
From: andrew at bemusement.org (Andrew Bennetts)
Date: Fri, 5 Mar 2010 15:17:59 +1100
Subject: [Python-ideas] iterable: next() and __iter__() -- and __reset()
In-Reply-To: <4B908217.9060603@gmx.net>
References: <20100304113553.514db067@o> <hmp9hn$tr3$1@dough.gmane.org>
	<4B908217.9060603@gmx.net>
Message-ID: <20100305041759.GH17708@steerpike.home.puzzling.org>

Mathias Panzenb?ck wrote:
> On 03/04/2010 10:43 PM, Georg Brandl wrote:
> >>>  (2) But: for any reason next() is not spelled as a "magic" method. If this
> >>>  method becomes the distinctive method of iterables, then it should be called
> >>>  __next__() for consistency. Side-question: Why is it called next(), as it is
> >>>  a magic method for iterators already?
> >Because it is supposed to be called directly.  __iter__() isn't.  (This changes
> >with Python 3, where you have next() as a builtin.)
> 
> And why is it made a builtin function? What was wrong with it being a normal method?

<http://www.python.org/dev/peps/pep-3114/>.

-Andrew.



From fuzzyman at gmail.com  Tue Mar 16 20:11:43 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Tue, 16 Mar 2010 19:11:43 +0000
Subject: [Python-ideas] EuroPython 2010 - Open for registration and reminder
	of participation
Message-ID: <6f4025011003161211r4c78da92q6d02f4ee676e7879@mail.gmail.com>

EuroPython 2010 - 17th to 24th July 2010
----------------------------------------
EuroPython is a conference for the Python programming language
community, including the Django, Zope and Plone communities. It is
aimed at everyone in the Python community, of all skill levels, both
users and programmers.

Last year's conference was the largest open source conference in the
UK and one of the largest community organised software conferences in
Europe.

This year EuroPython will be held from the 17th to 24th July in
Birmingham, UK. It will include over 100 talks, tutorials, sprints and
social events.

Registration
------------
Registration is open now at: http://www.europython.eu/registration/

For the best registration rates, book as soon as you can! Extra Early
Bird closes soon, after which normal Early Bird rate will apply until
10th May

 Talks, Activities and Events
----------------------------
Do you have something you wish to present at EuroPython? You want to
give a talk, run a tutorial or sprint?
Go to http://www.europython.eu/talks/cfp/ for information and advice!
Go to http://wiki.europython.eu/Sprints to plan a sprint!

Help Us Out
-----------
EuroPython is run by volunteers, like you! We could use a hand, and
any contribution is welcome.
Go to http://wiki.europython.eu/Helping to join us!
Go to http://www.europython.eu/contact/ to contact us directly!

Sponsors
--------
Sponsoring EuroPython is a unique opportunity to affiliate with this
prestigious conference and to reach a large number of Python users
from computing professionals to academics, from entrepreneurs to
motivated and well-educated job seekers.
http://www.europython.eu/sponsors/

Spread the Word
---------------
We are a community-run not-for-profit conference. Please help to
spread the word by distributing this announcement to colleagues,
project mailing lists, friends, your blog, Web site, and through your
social networking connections. Take a look at our publicity resources:
http://wiki.europython.eu/Publicity

 General Information
-------------------
For more information about the conference, please visit the official
site: http://www.europython.eu/

Looking forward to see you!
The EuroPython Team

-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100316/d2383e6e/attachment.html>

From stefan_ml at behnel.de  Wed Mar 17 08:11:29 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 17 Mar 2010 08:11:29 +0100
Subject: [Python-ideas] Matching multiple regex patterns simultaneously
In-Reply-To: <4B903D4B.1050209@canterbury.ac.nz>
References: <7659cab31003021239x76ba71cdo4c691f256776f60@mail.gmail.com>
	<4B903D4B.1050209@canterbury.ac.nz>
Message-ID: <hnpvb2$qfk$1@dough.gmane.org>

Greg Ewing, 05.03.2010 00:07:
> I have a half-finished project on the back burner
> to reimplement the core of Plex using Pyrex, but
> it's been languishing for so long that the burner
> has probably gone out by now. Maybe I should
> relight it...

Cython bootstraps the tiny truely performance-critical part of it 
(Scanners.py) into C code during installation and uses an external 
Scanners.pxd file to override the Python types in the module with C types. 
The other modules don't seem to matter in benchmarks, but compiling the 
inner loop of the scanner clearly gives a performance boost.

Stefan



From zac256 at gmail.com  Wed Mar 17 21:31:20 2010
From: zac256 at gmail.com (Zac Burns)
Date: Wed, 17 Mar 2010 13:31:20 -0700
Subject: [Python-ideas] __metafunc__
Message-ID: <333edbe81003171331m6ecfbffet90fb4d5b060e4ad6@mail.gmail.com>

I would like to propose a __metafunc__ that would work similar to
metaclasses but for functions.

My use case is for duck punching all functions defined in a module.  This
tends to be difficult as a postprocess after __import__ (you have to modify
method.im_func and deal with closures and functions in the module namespace
that might be references to functions defined elsewhere - it's a mess).

If instead when defining a function it did a lookup of __metafunc__ and
called it (like a decorator) after a function was defined then I could
import the module in a globals that includes my __metafunc__.

Feel free to tear this idea apart, improve it, or dash it on the rocks.
Perhaps some of you will also see use cases for this or something like it?

--
Zachary Burns
(407)590-4814
Aim - Zac256FL
Production Engineer (Digital Overlord)
Zindagi Games
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100317/ecb77afe/attachment.html>

From nbvfour at gmail.com  Thu Mar 18 07:49:39 2010
From: nbvfour at gmail.com (nbv4)
Date: Thu, 18 Mar 2010 02:49:39 -0400
Subject: [Python-ideas] setting function properties with a decorator syntax
In-Reply-To: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
Message-ID: <626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>

Decorators are great, they make this:

def some_func(*args, **kwargs):
    return "sup"
some_func = decorator(some_func)

into this:

@decorator
def some_func(*args, **kwargs):
    return "sup"

which makes the code look more structured. The line at the bottom
looks more like it 'belongs' to the function when it's at the top of
the def block, as opposed to the bottom.

But when it comes to function properties, it'd be nice if that same
pattern was availiable:

def some_func(*args, **kwargs):
    return "sup"
some_func.printable = True

becomes:

@printable=True
def some_func(*args, **kwargs):
    return "sup"

Any thoughts?


From pyideas at rebertia.com  Thu Mar 18 08:19:40 2010
From: pyideas at rebertia.com (Chris Rebert)
Date: Thu, 18 Mar 2010 00:19:40 -0700
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
Message-ID: <50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com>

On Wed, Mar 17, 2010 at 11:49 PM, nbv4 <nbvfour at gmail.com> wrote:
> Decorators are great, they make this:
>
> def some_func(*args, **kwargs):
> ? ?return "sup"
> some_func = decorator(some_func)
>
> into this:
>
> @decorator
> def some_func(*args, **kwargs):
> ? ?return "sup"
>
> which makes the code look more structured. The line at the bottom
> looks more like it 'belongs' to the function when it's at the top of
> the def block, as opposed to the bottom.
>
> But when it comes to function properties, it'd be nice if that same
> pattern was availiable:
>
> def some_func(*args, **kwargs):
> ? ?return "sup"
> some_func.printable = True
>
> becomes:
>
> @printable=True
> def some_func(*args, **kwargs):
> ? ?return "sup"
>
> Any thoughts?

Are function attributes used *that* much?

Cheers,
Chris
--
http://blog.rebertia.com


From anfedorov at gmail.com  Thu Mar 18 08:34:02 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Thu, 18 Mar 2010 03:34:02 -0400
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
Message-ID: <7659cab31003180034t5ef10b1fp1c663d74c9aae566@mail.gmail.com>

On Thu, Mar 18, 2010 at 2:49 AM, nbv4 <nbvfour at gmail.com> wrote:

> @printable=True
> def some_func(*args, **kwargs):
>    return "sup"
>
> Any thoughts?
>

If you have that many properties, there's no need to extend the syntax, just
do something like:

def prop(**props):
    def wee(f):
        f.__dict__.update(props)
        return f
    return wee

@prop(printable=True, other_prop="yo!")
def some_func():
    return "sup, %s" % some_func.other_prop

print some_func() # prints "sup, yo!"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100318/b99ffb24/attachment.html>

From jafo at tummy.com  Thu Mar 18 13:08:47 2010
From: jafo at tummy.com (Sean Reifschneider)
Date: Thu, 18 Mar 2010 06:08:47 -0600
Subject: [Python-ideas] Adding exception logging function to syslog module.
Message-ID: <20100318120847.GA8871@tummy.com>

As someone who writes mostly programs that run unattended on servers, one
thing I often want to do is have my tracebacks logged to syslog.  I would
propose adding something like the following to the syslog module:

   def logexceptions(also_stderr = True):
      class ExceptHook:
         def __init__(self, useStderr = False):
            self.useStderr = useStderr

         def __call__(self, etype, evalue, etb):
            import traceback, string
            tb = traceback.format_exception(*(etype, evalue, etb))
            tb = map(string.rstrip, tb)
            tb = string.join(tb, '\n')
            for line in string.split(tb, '\n'):
               syslog.syslog(line)
               if self.useStderr:
                  sys.stderr.write(line + '\n')
      sys.excepthook = ExceptHook(also_stderr)

Now, the downside to this is that currently the syslog module is entirely
in C.  So either I'd need to implement the above in C, or I'd need to add a
Python wrapper to the C module and use that.

This would allow users to also log exceptions to syslog by doing:

   import syslog
   syslog.openlog('myprogram', syslog.LOG_PID, syslog.LOG_MAIL)
   syslog.logexceptions()

Thoughts?

Thanks,
Sean
-- 
 Python: even though everyone's using it now, somehow it's still the coolest.
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability



From denis.spir at gmail.com  Thu Mar 18 19:32:05 2010
From: denis.spir at gmail.com (spir)
Date: Thu, 18 Mar 2010 19:32:05 +0100
Subject: [Python-ideas] setting function properties with a decorator
 syntax
In-Reply-To: <50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
	<50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com>
Message-ID: <20100318193205.0ffc7978@o>

On Thu, 18 Mar 2010 00:19:40 -0700
Chris Rebert <pyideas at rebertia.com> wrote:

> Are function attributes used *that* much?

I use them as a nice (& explicit, & clear) alternative to closures, esp. for func factories, eg:

def powerN(n):
    def f(x):
        return x ** f.n
    f.n = n
    return f
power3 = powerN(3)
for i in range(1,10):
    print "%s:%s" %(i,power3(i)),
# ==> 1:1 2:8 3:27 4:64 5:125 6:216 7:343 8:512 9:729

I find this conceptually much more satisfying than the following, since the parameter really is an attribute of the function (also, like a real upvalue, it can be explicetely updated).

def powerN(n):
    def f(x,n=n):	# ugly ;-)
        return x ** n
    return f

(Let us use the opportunity that python funcs are real objects :-)

A "parameterisable" generator factory:

def powers(n, start=1, stop=None):
    def f():
        while True:
            f.i += 1
            if f.stop and f.i >= f.stop:
                raise StopIteration
            yield (f.i, f.i ** f.n)
    f.n = n
    f.i = start-1
    f.stop = stop
    return f
for (i,x) in powers(3, 1,10)():
    print "%s:%s" %(i,x),
# ==> 1:1 2:8 3:27 4:64 5:125 6:216 7:343 8:512 9:729


Denis
________________________________

vit e estrany

spir.wikidot.com



From python at mrabarnett.plus.com  Thu Mar 18 19:56:43 2010
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 18 Mar 2010 18:56:43 +0000
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <20100318193205.0ffc7978@o>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>	<50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com>
	<20100318193205.0ffc7978@o>
Message-ID: <4BA2776B.8080206@mrabarnett.plus.com>

spir wrote:
> On Thu, 18 Mar 2010 00:19:40 -0700
> Chris Rebert <pyideas at rebertia.com> wrote:
> 
>> Are function attributes used *that* much?
> 
> I use them as a nice (& explicit, & clear) alternative to closures, esp. for func factories, eg:
> 
> def powerN(n):
>     def f(x):
>         return x ** f.n
>     f.n = n
>     return f
> power3 = powerN(3)
> for i in range(1,10):
>     print "%s:%s" %(i,power3(i)),
> # ==> 1:1 2:8 3:27 4:64 5:125 6:216 7:343 8:512 9:729
> 
> I find this conceptually much more satisfying than the following, since the parameter really is an attribute of the function (also, like a real upvalue, it can be explicetely updated).
> 
> def powerN(n):
>     def f(x,n=n):	# ugly ;-)
>         return x ** n
>     return f
> 
[snip]
This also works:

     def powerN(n):
         def f(x):
             return x ** n
         return f


From anfedorov at gmail.com  Thu Mar 18 22:01:55 2010
From: anfedorov at gmail.com (Andrey Fedorov)
Date: Thu, 18 Mar 2010 17:01:55 -0400
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <20100318193205.0ffc7978@o>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com> 
	<50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com> 
	<20100318193205.0ffc7978@o>
Message-ID: <7659cab31003181401r11580368t8a40468d4399254c@mail.gmail.com>

spir wrote:

> A "parameterisable" generator factory: [...]
>

Good lord, no! Nooo!

"power" is one thing with one semantic meaning
"range" is another thing with another semantic meaning


There is absolutely no excuse for mixing the two implementation into a
"power-range" monstrosity. Implement the two separately and make some custom
function composition like:

def pow(x, n):
    return x**n

def pow_o_range(n, a, b):
    return [(x, pow(x, n)) for x in range(a,b)]

for i, x in pow_o_range(3, 1, 10):
   print "%s:%s" % (i,x),

# prints "1:1 2:8 3:27 4:64 5:125 6:216 7:343 8:512 9:729"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100318/07984cb5/attachment.html>

From nbvfour at gmail.com  Thu Mar 18 23:02:52 2010
From: nbvfour at gmail.com (nbv4)
Date: Thu, 18 Mar 2010 15:02:52 -0700 (PDT)
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <7659cab31003181401r11580368t8a40468d4399254c@mail.gmail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com> 
	<50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com> 
	<20100318193205.0ffc7978@o>
	<7659cab31003181401r11580368t8a40468d4399254c@mail.gmail.com>
Message-ID: <6ddd86bc-2aa4-4a98-bf4f-d87bfb771db1@q16g2000yqq.googlegroups.com>

On Mar 18, 5:01?pm, Andrey Fedorov <anfedo... at gmail.com> wrote:
> spir wrote:
> > A "parameterisable" generator factory: [...]
>
> Good lord, no! Nooo!
>
> "power" is one thing with one semantic meaning
> "range" is another thing with another semantic meaning
>
> There is absolutely no excuse for mixing the two implementation into a
> "power-range" monstrosity. Implement the two separately and make some custom
> function composition like:
>
> def pow(x, n):
> ? ? return x**n
>
> def pow_o_range(n, a, b):
> ? ? return [(x, pow(x, n)) for x in range(a,b)]
>
> for i, x in pow_o_range(3, 1, 10):
> ? ?print "%s:%s" % (i,x),
>
> # prints "1:1 2:8 3:27 4:64 5:125 6:216 7:343 8:512 9:729"
>

i was just an example to demonstrate the usage dude, :)


From greg.ewing at canterbury.ac.nz  Fri Mar 19 02:50:56 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 19 Mar 2010 14:50:56 +1300
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <20100318193205.0ffc7978@o>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
	<50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com>
	<20100318193205.0ffc7978@o>
Message-ID: <4BA2D880.9010908@canterbury.ac.nz>

spir wrote:

> def powerN(n):
>     def f(x,n=n):	# ugly ;-)
>         return x ** n
>     return f

Since nothing can change the value of n there after powerN
returns, there's no need for default argument abuse. You
can just write

   def powerN(n):
      def f(x):
        return x ** n
      return f

-- 
Greg


From dstanek at dstanek.com  Fri Mar 19 03:28:46 2010
From: dstanek at dstanek.com (David Stanek)
Date: Thu, 18 Mar 2010 22:28:46 -0400
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
Message-ID: <de32cc031003181928i467101c1nbfe0b7d715c3ed59@mail.gmail.com>

On Thu, Mar 18, 2010 at 2:49 AM, nbv4 <nbvfour at gmail.com> wrote:
>
> @printable=True
> def some_func(*args, **kwargs):
> ? ?return "sup"
>
> Any thoughts?
>

I don't remember seeing much code using function properties. I would
probably just create a new decorator for this.

    >>> def add_props(**kwargs):
    ...     def _(func):
    ...         for key in kwargs:
    ...             setattr(func, key, kwargs[key])
    ...         return func
    ...     return _
    ...
    >>> @add_props(printable=True)
    ... def f():
    ...     """Example function"""
    ...
    >>> f.printable
    True
    >>>


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek


From fdrake at acm.org  Fri Mar 19 03:33:40 2010
From: fdrake at acm.org (Fred Drake)
Date: Thu, 18 Mar 2010 22:33:40 -0400
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <de32cc031003181928i467101c1nbfe0b7d715c3ed59@mail.gmail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com> 
	<de32cc031003181928i467101c1nbfe0b7d715c3ed59@mail.gmail.com>
Message-ID: <9cee7ab81003181933g37d88a82g973c4d8b3d4369e7@mail.gmail.com>

On Thu, Mar 18, 2010 at 10:28 PM, David Stanek <dstanek at dstanek.com> wrote:
> I don't remember seeing much code using function properties. I would
> probably just create a new decorator for this.

I've not either, but if we determine that this is a useful facility to
include in the standard library, I'd be in favor of something like
this:

@setattr(printable=True)
def f():
    """Example function"""


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller


From scott+python-ideas at scottdial.com  Fri Mar 19 04:43:34 2010
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Thu, 18 Mar 2010 23:43:34 -0400
Subject: [Python-ideas] setting function properties with a
	decorator	syntax
In-Reply-To: <9cee7ab81003181933g37d88a82g973c4d8b3d4369e7@mail.gmail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
	<de32cc031003181928i467101c1nbfe0b7d715c3ed59@mail.gmail.com>
	<9cee7ab81003181933g37d88a82g973c4d8b3d4369e7@mail.gmail.com>
Message-ID: <4BA2F2E6.9020204@scottdial.com>

On 3/18/2010 10:33 PM, Fred Drake wrote:
> On Thu, Mar 18, 2010 at 10:28 PM, David Stanek <dstanek at dstanek.com> wrote:
>> I don't remember seeing much code using function properties. I would
>> probably just create a new decorator for this.
> 
> I've not either, but if we determine that this is a useful facility to
> include in the standard library, I'd be in favor of something like
> this:
> 
> @setattr(printable=True)
> def f():
>     """Example function"""
> 

For the sake of a head count, I have used function attributes for the
purpose of memoizing computations (both return values and intermediate
calculations). A representative example:

@setattr(primes=set(), composites=set())
def is_prime(n):
    if n <= 0:
        raise ValueError(n)
    elif n == 1:
        return True
    elif n in is_prime.primes:
        return True
    elif n in is_prime.composites:
        return False
    for p in is_prime.primes:
        if n % p == 0:
            is_prime.composites.add(n)
            return False
    is_prime.primes.add(n)
    return True

But honestly, the biggest problem I have with this pattern is that you
explicitly have to say the name of the function since there is no "self"
equivalent. I think it would be more maintenance-friendly to do:

def _(n):
    primes = set()
    composites = set()
    def is_prime(n):
        if n <= 0:
            raise ValueError(n)
        elif n == 1:
            return True
        elif n in primes:
            return True
        elif n in composites:
            return False
        for p in primes:
            if n % p == 0:
                composites.add(n)
                return False
        primes.add(n)
        return True
    return is_prime
is_prime = _()

Which is to say, I would be more amenable to adding such a feature if it
was building closure. And in that usage, it would seem to be more
generally useful and well-behaved.. outsiders don't normally modify
closures (if you want this sort of interface, you probably should define
a class).. it allows you to avoid naming the function in the function..
and it is a much closer substitute for the default arguments kludge that
is so commonly abused.

@closure(primes=set(), composites=set())
def is_prime(n):
    if n <= 0:
        raise ValueError(n)
    elif n == 1:
        return True
    elif n in primes:
        return True
    elif n in composites:
        return False
    for p in primes:
        if n % p == 0:
            composites.add(n)
            return False
    primes.add(n)
    return True

But, that's just my two cents.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From arnodel at googlemail.com  Fri Mar 19 09:25:09 2010
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Fri, 19 Mar 2010 08:25:09 +0000
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <4BA2F2E6.9020204@scottdial.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
	<de32cc031003181928i467101c1nbfe0b7d715c3ed59@mail.gmail.com>
	<9cee7ab81003181933g37d88a82g973c4d8b3d4369e7@mail.gmail.com>
	<4BA2F2E6.9020204@scottdial.com>
Message-ID: <B6A62117-E42B-4D88-8F78-DD93E1FE0963@googlemail.com>



On 19 Mar 2010, at 03:43, Scott Dial <scott+python- 
ideas at scottdial.com> wrote:

> On 3/18/2010 10:33 PM, Fred Drake wrote:
>> On Thu, Mar 18, 2010 at 10:28 PM, David Stanek  
>> <dstanek at dstanek.com> wrote:
>>> I don't remember seeing much code using function properties. I would
>>> probably just create a new decorator for this.
>>
>> I've not either, but if we determine that this is a useful facility  
>> to
>> include in the standard library, I'd be in favor of something like
>> this:
>>
>> @setattr(printable=True)
>> def f():
>>    """Example function"""
>>
>
> For the sake of a head count, I have used function attributes for the
> purpose of memoizing computations (both return values and intermediate
> calculations). A representative example:
>
> @setattr(primes=set(), composites=set())
> def is_prime(n):
>    if n <= 0:
>        raise ValueError(n)
>    elif n == 1:
>        return True
>    elif n in is_prime.primes:
>        return True
>    elif n in is_prime.composites:
>        return False
>    for p in is_prime.primes:
>        if n % p == 0:
>            is_prime.composites.add(n)
>            return False
>    is_prime.primes.add(n)
>    return True
>
> But honestly, the biggest problem I have with this pattern is that you
> explicitly have to say the name of the function since there is no  
> "self"
> equivalent. I think it would be more maintenance-friendly to do:
>
> def _(n):
>    primes = set()
>    composites = set()
>    def is_prime(n):
>        if n <= 0:
>            raise ValueError(n)
>        elif n == 1:
>            return True
>        elif n in primes:
>            return True
>        elif n in composites:
>            return False
>        for p in primes:
>            if n % p == 0:
>                composites.add(n)
>                return False
>        primes.add(n)
>        return True
>    return is_prime
> is_prime = _()
>
> Which is to say, I would be more amenable to adding such a feature  
> if it
> was building closure. And in that usage, it would seem to be more
> generally useful and well-behaved.. outsiders don't normally modify
> closures (if you want this sort of interface, you probably should  
> define
> a class).. it allows you to avoid naming the function in the  
> function..
> and it is a much closer substitute for the default arguments kludge  
> that
> is so commonly abused.
>
> @closure(primes=set(), composites=set())
> def is_prime(n):
>    if n <= 0:
>        raise ValueError(n)
>    elif n == 1:
>        return True
>    elif n in primes:
>        return True
>    elif n in composites:
>        return False
>    for p in primes:
>        if n % p == 0:
>            composites.add(n)
>            return False
>    primes.add(n)
>    return True
>
> But, that's just my two cents.
>
> -- 
> Scott Dial
> scott at scottdial.com
> scodial at cs.indiana.edu
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas

Why not just use default arguments?

Arnaud


From scott+python-ideas at scottdial.com  Fri Mar 19 10:07:29 2010
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Fri, 19 Mar 2010 05:07:29 -0400
Subject: [Python-ideas] setting function properties with a
	decorator	syntax
In-Reply-To: <B6A62117-E42B-4D88-8F78-DD93E1FE0963@googlemail.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>	<de32cc031003181928i467101c1nbfe0b7d715c3ed59@mail.gmail.com>	<9cee7ab81003181933g37d88a82g973c4d8b3d4369e7@mail.gmail.com>	<4BA2F2E6.9020204@scottdial.com>
	<B6A62117-E42B-4D88-8F78-DD93E1FE0963@googlemail.com>
Message-ID: <4BA33ED1.2090409@scottdial.com>

On 3/19/2010 4:25 AM, Arnaud Delobelle wrote:
> On 19 Mar 2010, at 03:43, Scott Dial <scott+python-ideas at scottdial.com>
> wrote:
>> Which is to say, I would be more amenable to adding such a feature if it
>> was building closure. And in that usage, it would seem to be more
>> generally useful and well-behaved.. outsiders don't normally modify
>> closures (if you want this sort of interface, you probably should define
>> a class).. it allows you to avoid naming the function in the function..
>> and it is a much closer substitute for the default arguments kludge that
>> is so commonly abused.
>>
> 
> Why not just use default arguments?
> 

My problem with this particular kludge is that rarely are these
arguments meaningful as *arguments*. They become a part of the function
definition that shows up in documentation. Worse to me is that the
keyword names block those names from appearing in **kwds, which is
problematic for creating a transparent function (e.g., when writing a
decorator that takes an unknown set of keyword arguments) and evokes
dynamic-scoping madness. Additionally, they are a source of subtle
errors when extra arguments are passed to functions (although this is
mitigated mostly by Py3's keyword-only arguments syntax).

All of those problems go away with a magical @closure() decorator, but I
realize that creating such a decorator would be quite a kludge
in-and-of-itself. Perhaps it's a bit too clever.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From arnodel at googlemail.com  Fri Mar 19 14:15:20 2010
From: arnodel at googlemail.com (Arnaud Delobelle)
Date: Fri, 19 Mar 2010 13:15:20 +0000
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <4BA33ED1.2090409@scottdial.com>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
	<de32cc031003181928i467101c1nbfe0b7d715c3ed59@mail.gmail.com>
	<9cee7ab81003181933g37d88a82g973c4d8b3d4369e7@mail.gmail.com>
	<4BA2F2E6.9020204@scottdial.com>
	<B6A62117-E42B-4D88-8F78-DD93E1FE0963@googlemail.com>
	<4BA33ED1.2090409@scottdial.com>
Message-ID: <9bfc700a1003190615u721d0e94x12875914c61cf9a6@mail.gmail.com>

On 19 March 2010 09:07, Scott Dial <scott+python-ideas at scottdial.com> wrote:
> On 3/19/2010 4:25 AM, Arnaud Delobelle wrote:
>> On 19 Mar 2010, at 03:43, Scott Dial <scott+python-ideas at scottdial.com>
>> wrote:
>>> Which is to say, I would be more amenable to adding such a feature if it
>>> was building closure. And in that usage, it would seem to be more
>>> generally useful and well-behaved.. outsiders don't normally modify
>>> closures (if you want this sort of interface, you probably should define
>>> a class).. it allows you to avoid naming the function in the function..
>>> and it is a much closer substitute for the default arguments kludge that
>>> is so commonly abused.
>>>
>>
>> Why not just use default arguments?
>>
>
> My problem with this particular kludge is that rarely are these
> arguments meaningful as *arguments*. They become a part of the function
> definition that shows up in documentation. Worse to me is that the
> keyword names block those names from appearing in **kwds, which is
> problematic for creating a transparent function (e.g., when writing a
> decorator that takes an unknown set of keyword arguments) and evokes
> dynamic-scoping madness. Additionally, they are a source of subtle
> errors when extra arguments are passed to functions (although this is
> mitigated mostly by Py3's keyword-only arguments syntax).
>
> All of those problems go away with a magical @closure() decorator, but I
> realize that creating such a decorator would be quite a kludge
> in-and-of-itself. Perhaps it's a bit too clever.

I wrote such a decorator a while ago, I called it 'bind'.  It was
indeed quite a kludge, but if you're interested I can post it.  It
worked by rewriting the bytecode of the function it decorated.  If I
remember correctly, it has some limitations, e.g. it didn't work with
local functions i.e.

@bind(x=3)
def f():
    def g():
        return x
    return g

f()()

would return the current value of the global variable 'x' rather than
3.  I don't think I got round to solving this (it would require
looking at the constants in the code which are themselves code objects
an apply the same bytecode transformation recursively).

Also, it was for Python 2.X so I don't know if it would work as-is in Python 3.

-- 
Arnaud


From jimjjewett at gmail.com  Fri Mar 19 15:57:42 2010
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 19 Mar 2010 10:57:42 -0400
Subject: [Python-ideas] setting function properties with a decorator
	syntax
In-Reply-To: <20100318193205.0ffc7978@o>
References: <f98de9ae-8653-4693-85a3-739319f72a51@q23g2000yqd.googlegroups.com>
	<626451591003172349q307789f3v43c62def59692c7c@mail.gmail.com>
	<50697b2c1003180019u768b50anb650293376d6a88f@mail.gmail.com>
	<20100318193205.0ffc7978@o>
Message-ID: <fb6fbf561003190757n6c9f3c11i3d32b155b6f5b3d7@mail.gmail.com>

For anyone finding this on a later search, note that he still uses a
closure; just a slightly neater one.

Within a function, there is no way to refer to the function itself,
except by name -- and that name may well have been reassigned to
something else.  A more local enclosure protects against that
reassignment.

-jJ

On 3/18/10, spir <denis.spir at gmail.com> wrote:
> On Thu, 18 Mar 2010 00:19:40 -0700
> Chris Rebert <pyideas at rebertia.com> wrote:
>
>> Are function attributes used *that* much?
>
> I use them as a nice (& explicit, & clear) alternative to closures, esp. for
> func factories, eg:
>
> def powerN(n):
>     def f(x):
>         return x ** f.n
>     f.n = n
>     return f
> power3 = powerN(3)
> for i in range(1,10):
>     print "%s:%s" %(i,power3(i)),
> # ==> 1:1 2:8 3:27 4:64 5:125 6:216 7:343 8:512 9:729
>
> I find this conceptually much more satisfying than the following, since the
> parameter really is an attribute of the function (also, like a real upvalue,
> it can be explicetely updated).
>
> def powerN(n):
>     def f(x,n=n):	# ugly ;-)
>         return x ** n
>     return f
>
> (Let us use the opportunity that python funcs are real objects :-)
>
> A "parameterisable" generator factory:
>
> def powers(n, start=1, stop=None):
>     def f():
>         while True:
>             f.i += 1
>             if f.stop and f.i >= f.stop:
>                 raise StopIteration
>             yield (f.i, f.i ** f.n)
>     f.n = n
>     f.i = start-1
>     f.stop = stop
>     return f
> for (i,x) in powers(3, 1,10)():
>     print "%s:%s" %(i,x),
> # ==> 1:1 2:8 3:27 4:64 5:125 6:216 7:343 8:512 9:729
>
>
> Denis
> ________________________________
>
> vit e estrany
>
> spir.wikidot.com
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>


From eric at trueblade.com  Tue Mar 23 00:31:08 2010
From: eric at trueblade.com (Eric Smith)
Date: Mon, 22 Mar 2010 19:31:08 -0400
Subject: [Python-ideas] Adding exception logging function to syslog
	module.
In-Reply-To: <20100318120847.GA8871@tummy.com>
References: <20100318120847.GA8871@tummy.com>
Message-ID: <4BA7FDBC.3060808@trueblade.com>

Sean Reifschneider wrote:
> As someone who writes mostly programs that run unattended on servers, one
> thing I often want to do is have my tracebacks logged to syslog.  I would
> propose adding something like the following to the syslog module:
> 
>    def logexceptions(also_stderr = True):
>       class ExceptHook:
>          def __init__(self, useStderr = False):
>             self.useStderr = useStderr
> 
>          def __call__(self, etype, evalue, etb):
>             import traceback, string
>             tb = traceback.format_exception(*(etype, evalue, etb))
>             tb = map(string.rstrip, tb)
>             tb = string.join(tb, '\n')
>             for line in string.split(tb, '\n'):
>                syslog.syslog(line)
>                if self.useStderr:
>                   sys.stderr.write(line + '\n')
>       sys.excepthook = ExceptHook(also_stderr)
> 
> Now, the downside to this is that currently the syslog module is entirely
> in C.  So either I'd need to implement the above in C, or I'd need to add a
> Python wrapper to the C module and use that.
> 
> This would allow users to also log exceptions to syslog by doing:
> 
>    import syslog
>    syslog.openlog('myprogram', syslog.LOG_PID, syslog.LOG_MAIL)
>    syslog.logexceptions()
> 
> Thoughts?

I think this would be a great addition. I have to do similar things all 
of the time, although I never though of using sys.excepthook, for some 
reason. I'll have to steal your code in the meantime!

A simpler pure Python version would be:

from __future__ import print_function

def logexceptions(also_stderr=True):
     import sys
     import traceback
     import syslog
     def syslog_exception(etype, evalue, etb):
         # The result of traceback.format_exception might contain 
 

         # embedded newlines, so we have the nested loops. 
 

         for line in traceback.format_exception(etype, evalue, etb):
             for line in line.rstrip().split('\n'):
                 syslog.syslog(line)
                 if also_stderr:
                     print(line, file=sys.stderr)
     sys.excepthook = syslog_exception

logexceptions(True)

If you need any help implementing this in C, I'll assist (although I'm 
out of town for 2 weeks starting Thursday).

-- 
Eric.


From eric at trueblade.com  Tue Mar 23 21:24:41 2010
From: eric at trueblade.com (Eric Smith)
Date: Tue, 23 Mar 2010 16:24:41 -0400
Subject: [Python-ideas] Adding exception logging function to
	syslog	module.
In-Reply-To: <4BA7FDBC.3060808@trueblade.com>
References: <20100318120847.GA8871@tummy.com> <4BA7FDBC.3060808@trueblade.com>
Message-ID: <4BA92389.7060401@trueblade.com>

Eric Smith wrote:
> I think this would be a great addition. I have to do similar things all 
> of the time, although I never though of using sys.excepthook, for some 
> reason. I'll have to steal your code in the meantime!
> 
> A simpler pure Python version would be:
> 
> from __future__ import print_function
> 
> def logexceptions(also_stderr=True):
>     import sys
>     import traceback
>     import syslog
>     def syslog_exception(etype, evalue, etb):
>         # The result of traceback.format_exception might contain
> 
>         # embedded newlines, so we have the nested loops.
> 
>         for line in traceback.format_exception(etype, evalue, etb):
>             for line in line.rstrip().split('\n'):
>                 syslog.syslog(line)
>                 if also_stderr:
>                     print(line, file=sys.stderr)
>     sys.excepthook = syslog_exception
> 
> logexceptions(True)

On second thought, maybe it would be better to optionally chain to the 
existing sys.excepthook instead of assuming it writes to stderr. The 
above code would become:

def logexceptions(chain=True):
     import sys
     import traceback
     import syslog
     current_hook = sys.excepthook
     def syslog_exception(etype, evalue, etb):
         if chain:
             current_hook(etype, evalue, etb)
         # The result of traceback.format_exception might contain 
 

         # embedded newlines, so we have the nested loops. 
 

         for line in traceback.format_exception(etype, evalue, etb):
             for line in line.rstrip().split('\n'):
                 syslog.syslog(line)
     sys.excepthook = syslog_exception

I think I'll open an issue for this. I don't think the language 
moratorium would preclude its addition.

Eric.


-- 
Eric.


From jafo at tummy.com  Wed Mar 24 22:57:42 2010
From: jafo at tummy.com (Sean Reifschneider)
Date: Wed, 24 Mar 2010 15:57:42 -0600
Subject: [Python-ideas] Adding exception logging function to
	syslog	module.
In-Reply-To: <4BA92389.7060401@trueblade.com>
References: <20100318120847.GA8871@tummy.com> <4BA7FDBC.3060808@trueblade.com>
	<4BA92389.7060401@trueblade.com>
Message-ID: <4BAA8AD6.3020807@tummy.com>

On 03/23/2010 02:24 PM, Eric Smith wrote:
> On second thought, maybe it would be better to optionally chain to the
> existing sys.excepthook instead of assuming it writes to stderr. The
> above code would become:

That looks just great.  I understand you're leaving tomorrow, so I'll
probably try converting it to C while you're away.  I expect I should be
able to find some time.  I'll make an issue for it if you haven't already,
and submit it for review there.

I don't imagine it will be too hard, but I haven't really done any C API
hacking at the level of creating an inner function.  I should be able to
figure it out though.

Thanks, and have a good trip.

Sean
-- 
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100324/6efc280d/attachment.pgp>

From eric at trueblade.com  Wed Mar 24 23:24:46 2010
From: eric at trueblade.com (Eric Smith)
Date: Wed, 24 Mar 2010 18:24:46 -0400
Subject: [Python-ideas] Adding exception logging function
	to	syslog	module.
In-Reply-To: <4BAA8AD6.3020807@tummy.com>
References: <20100318120847.GA8871@tummy.com>
	<4BA7FDBC.3060808@trueblade.com>	<4BA92389.7060401@trueblade.com>
	<4BAA8AD6.3020807@tummy.com>
Message-ID: <4BAA912E.7010501@trueblade.com>

Sean Reifschneider wrote:
> On 03/23/2010 02:24 PM, Eric Smith wrote:
>> On second thought, maybe it would be better to optionally chain to the
>> existing sys.excepthook instead of assuming it writes to stderr. The
>> above code would become:
> 
> That looks just great.  I understand you're leaving tomorrow, so I'll
> probably try converting it to C while you're away.  I expect I should be
> able to find some time.  I'll make an issue for it if you haven't already,
> and submit it for review there.
> 
> I don't imagine it will be too hard, but I haven't really done any C API
> hacking at the level of creating an inner function.  I should be able to
> figure it out though.
> 
> Thanks, and have a good trip.

Thanks, Sean.

It's issue 8214, you should already be nosy on it. I used a slightly 
different method there, designed to remove a reference to sys.excepthook 
if not chaining, but the principle is the same.

I'm not sure about the closure aspect of this in C. You could do it with 
a global variable, but I'm not wild about that. Maybe you can ask on 
python-dev for suggestions.

I'll be checking email occasionally while I'm away.

-- 
Eric.


From jafo at tummy.com  Wed Mar 24 23:34:52 2010
From: jafo at tummy.com (Sean Reifschneider)
Date: Wed, 24 Mar 2010 16:34:52 -0600
Subject: [Python-ideas] Adding exception logging function
	to	syslog	module.
In-Reply-To: <4BAA912E.7010501@trueblade.com>
References: <20100318120847.GA8871@tummy.com>
	<4BA7FDBC.3060808@trueblade.com>	<4BA92389.7060401@trueblade.com>
	<4BAA8AD6.3020807@tummy.com> <4BAA912E.7010501@trueblade.com>
Message-ID: <4BAA938C.7000003@tummy.com>

On 03/24/2010 04:24 PM, Eric Smith wrote:
> I'm not sure about the closure aspect of this in C. You could do it with
> a global variable, but I'm not wild about that. Maybe you can ask on
> python-dev for suggestions.

I'll probably do some digging into the API to see about creating a function
via C, and from that the issue of the closure will probably become more
clear.

If not, python-dev will be my go-to-guy.  :-)

Sean
-- 
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100324/77290cd3/attachment.pgp>

From alexander.belopolsky at gmail.com  Thu Mar 25 07:50:41 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 25 Mar 2010 02:50:41 -0400
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<AB3214E2-46DC-44FD-8545-6C3FF08FA88D@gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
Message-ID: <d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>

Since we are now venturing in the realm of python-ideas, I am
switching the mailing lists.  Please see more below.

On Wed, Mar 24, 2010 at 7:31 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Wed, Mar 24, 2010 at 11:11 PM, Alexander Belopolsky
> <alexander.belopolsky at gmail.com> wrote:
>> On Wed, Mar 24, 2010 at 7:02 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> ..
>>>
>>> So if I understand correctly, you propose that float('nan') ==
>>> float('nan') return True. ?Would you also suggest extending this
>>> behaviour to Decimal nans?
>>>
>>
>> yes
>>
>
> Okay. ?So now it seems to me that there are many decisions to make:

I maybe show my ignorance, but I don't see the significance of
bringing Decimal nans to the discussion.  My answers below apply
equally to binary and decimal floating point numbers.

> should any Decimal nan compare equal to any other?

Yes.

>?What if the two nans have different payloads or signs?

They are still equal.  Just as 0.0 and -0.0 are now.

>?How about comparing a
> signaling nan with either an arbitrary quiet nan, or with the exact
> quiet nan that corresponds to the signaling nan?

I don't think signaling nans can be produced using Python float type.
I may be mistaken about this and the answer may be different in some
decimal contexts.  Nevertheless, I always thought that any operation
with an sNaN should raise an exception, so comparing an sNaN with
anything, NaN or not, should just raise an exception.


> ?How do decimal nans compare with float nans?

Good question.  Does IEEE 754 or rather 854 specify cross-radix
comparisons?  My answer would be that they should compare equal.

> ?Should Decimal.compare(Decimal('nan'), Decimal('nan')) return 0 rather than nan?

I would think with deprecation of __cmp__, Decimal.compare would not
be that useful.  I would rather not expand this discussion to what NaN
< NaN should return.  In my view it should just raise an exception
because there is no practical way to propagate NaN through boolean
results.  Decimal.compare may similarly raise an exception when
comparing NaNs.  Returning Decimal('nan') seems least useful.  With
the current NaN properties, I would rather see Decimal.compare return
2 for unordered values.

>?If not, how do you justify the difference between == and compare?

Just as  (a > b or b < a) is currently not equivalent to a != b in the
presence of unordered values (i.e. when a and b are of different
type).

> ?If so, how do you justify the
> deviation from the standard on which the decimal modulo is based?

I assume you mean "decimal module" because I don't know what "decimal
modulo" is based on.  Is that IEEE 854?  I need to get a copy of that
before I can answer. :-)

>
> In answering all these questions, you effectively end up developing
> your own standard, and hoping that all the answers you chose are
> sensible, consistent, and useful.
>

As Grace Hopper is reported to have said, "The wonderful thing about
standards is that there are so many of them to choose from."  As much
as I don't like most of the choices that Java designers made, I think
they got this one right.

> Alternatively, we could do what we're currently doing: ?make use of
> *existing* standards to answer these questions, and rely on the
> expertise of the many who've thought about this in depth.
>


I don't think I can answer this better than Bertrand Meyer:

"""
Before commenting further let me note the obvious: I am by no means a
numerics expert; I know that IEEE 754 was a tremendous advance, and
that it was designed by some of the best minds in the field, headed by
Velvel Kahan who received a Turing Award in part for that success. So
it is quite possible that I am about to bury myself in piles of
ridicule. On the other hand I have also learned that (1) ridicule does
not kill, so I am game; and more importantly (2) experts are often
right but not always, and it is always proper to question their
reasoning. So without fear let me not stop at the arguments that ?the
committee must have voted on this point and they obviously knew what
they were doing? and ?it is the standard and implemented on zillions
of machines, you cannot change it now?. Both are true enough, but not
an excuse to censor questions.
"""
http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/


> You say that you don't see any compromise: ?I say that there's value
> in adhering to (de facto and de jure) standards, and I see a
> compromise between standards adherence and Python pragmatics.

Bertrand Meyer again:

"""
A few of us who had to examine the issue recently think that ?
whatever the standard says at the machine level ? a programming
language should support the venerable properties that equality is
reflexive and that assignment yields equality.
"""

IEEE standards were developed in a different problem domain: hardware
or low level programming language design.   They may not be
appropriate for an object oriented language like Python.  Java and
recently Eiffel designers seem to  have realized that.


From denis.spir at gmail.com  Thu Mar 25 11:48:00 2010
From: denis.spir at gmail.com (spir =?UTF-8?B?4pij?=)
Date: Thu, 25 Mar 2010 11:48:00 +0100
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<AB3214E2-46DC-44FD-8545-6C3FF08FA88D@gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
Message-ID: <20100325114800.072bfa1a@o>

On Thu, 25 Mar 2010 02:50:41 -0400
Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:

> Bertrand Meyer again:
> 
> """
> A few of us who had to examine the issue recently think that ?
> whatever the standard says at the machine level ? a programming
> language should support the venerable properties that equality is
> reflexive and that assignment yields equality.
> """
> 
> IEEE standards were developed in a different problem domain: hardware
> or low level programming language design.   They may not be
> appropriate for an object oriented language like Python.  Java and
> recently Eiffel designers seem to  have realized that.

Hum, should the above be interpreted as:
	a = NAN
==> not only
	a is NAN
but also
	a == NAN

and further:
	b = a
==>
	b == a

?
(Else there should be a distinction between equality assignment and identity assignemt?
	b = a	# ==> a is b and a == b
	b := a	# ==> a is b and possibly a == b
 ;-)

Denis
________________________________

vit esse estrany ?

spir.wikidot.com


From masklinn at masklinn.net  Thu Mar 25 12:28:22 2010
From: masklinn at masklinn.net (Masklinn)
Date: Thu, 25 Mar 2010 12:28:22 +0100
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<AB3214E2-46DC-44FD-8545-6C3FF08FA88D@gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
Message-ID: <83DAEA2C-05CF-45EC-BCE7-0B30490DDEC1@masklinn.net>

On 25 Mar 2010, at 07:50 , Alexander Belopolsky wrote:
> 
> Since we are now venturing in the realm of python-ideas, I am
> switching the mailing lists.  Please see more below.
> 
> On Wed, Mar 24, 2010 at 7:31 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> On Wed, Mar 24, 2010 at 11:11 PM, Alexander Belopolsky
>> <alexander.belopolsky at gmail.com> wrote:
>>> On Wed, Mar 24, 2010 at 7:02 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>> ..
>>>> 
>>>> So if I understand correctly, you propose that float('nan') ==
>>>> float('nan') return True.  Would you also suggest extending this
>>>> behaviour to Decimal nans?
>>>> 
>>> 
>>> yes
>>> 
>> 
>> Okay.  So now it seems to me that there are many decisions to make:
> 
> I maybe show my ignorance, but I don't see the significance of
> bringing Decimal nans to the discussion.  My answers below apply
> equally to binary and decimal floating point numbers.
> 
>> should any Decimal nan compare equal to any other?
> 
> Yes.
> 
>>  What if the two nans have different payloads or signs?
> 
> They are still equal.  Just as 0.0 and -0.0 are now.

Excuse me but NaNs propagate right? So we'd have not only
float('nan') == float('nan'), but also float('nan') = float('nan') + 1
right? Indeed, any operation on a NaN would return a result equal (and
identical?) to itself. I'm not sure that makes more sense than the current
behavior (which is at least standard and specified).

> IEEE standards were developed in a different problem domain: hardware
> or low level programming language design.

I don't really agree. They were developed in a problem domain of mapping
abstract mathematical concepts to computing. And it's not like they're 
static and never updated (indeed, IEEE-754 was updated in 2008, if anyone
has an IEEE login it would be possible to check the purposes and
assumptions the standard set for itself).

I believe changing the behavior of nan to have "nan == nan" would be
broken, nan is not a "normal" value (or set of values) of the space.

Having the option of getting an exception upon getting a nan on the other
hand (as the default behavior even, with a switch to the current behavior
using contexts similar to decimal's), why not.

> http://bertrandmeyer.com/2010/02/06/reflexivity-and-other-pillars-of-civilization/

It's interesting that Bertrand starts from

> I am talking about a value, as in mathematics, not an expression, as in programming

but doesn't seem to consider that, in the first place, the very concept of
NaN is due to the limitations of physical space compared to the abstract
mathematical concepts involved. The very idea of IEEE-754 floats are a
a definite departure from the fundamental laws of mathematics, making
Bertrand's final request

> It is rather dangerous indeed to depart from the fundamental laws of mathematics. 

very weird. Unless you remove floats from your language, you're not dealing
with grounds mathematics cover in the first place, you're dealing with
a concretization of mathematical ideas, and as all changes in abstraction
levels, it will leak one way or another.

I'd support "fixing" nan behaviors, but not by making operations on nan
reflexive (and nan a normal value, which it isn't) and instead by
treating all of them as errors, the same way calling an arbitrary
attribute no None results in an AttributeError in Python.

From alexander.belopolsky at gmail.com  Thu Mar 25 19:06:47 2010
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Thu, 25 Mar 2010 14:06:47 -0400
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
Message-ID: <d38f5331003251106x71c82462n7161d8bc7640a9f7@mail.gmail.com>

> On Wed, Mar 24, 2010 at 7:31 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
..
>>?What if the two nans have different payloads or signs?
>
> They are still equal. ?Just as 0.0 and -0.0 are now.
>

Interestingly, Java departs from IEEE 754 on that last point as well:

"""
Note that in most cases, for two instances of class Double, d1 and d2,
the value of d1.equals(d2) is true if and only if

   d1.doubleValue() == d2.doubleValue()

also has the value true. However, there are two exceptions:

If d1 and d2 both represent Double.NaN, then the equals method returns
true, even though Double.NaN==Double.NaN has the value false.
If d1 represents +0.0 while d2 represents -0.0, or vice versa, the
equal test has the value false, even though +0.0==-0.0 has the value
true. This allows hashtables to operate properly.
""" http://java.sun.com/j2se/1.3/docs/api/java/lang/Double.html


From rhamph at gmail.com  Thu Mar 25 19:33:48 2010
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 25 Mar 2010 12:33:48 -0600
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
Message-ID: <aac2c7cb1003251133o539f5dbej2616c6cabe7547f4@mail.gmail.com>

IMO, we should go for the simplest of all usage: a NaN singleton,
shared by float and Decimal, compares equal to itself, unsortable, no
payload.

Usage of the payload is hypothetical.  Situations where comparing
false is "more correct" than comparing true are hypothetical, and
grossly outweighed by the situations where comparing true is blatantly
correct.

IEEE 754 doesn't say NaN should compare false; it says it should
compare "unordered".  We can't do that as we're not using a 4-way
comparison (which wouldn't generalize to complex anyway), so we have
to make up our own solution.  It might as well fit our needs, rather
than nobodies.


-- 
Adam Olsen, aka Rhamphoryncus


From dickinsm at gmail.com  Thu Mar 25 19:58:12 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 25 Mar 2010 18:58:12 +0000
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <aac2c7cb1003251133o539f5dbej2616c6cabe7547f4@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<aac2c7cb1003251133o539f5dbej2616c6cabe7547f4@mail.gmail.com>
Message-ID: <5c6f2a5d1003251158j850da20ka310d24c1f6be819@mail.gmail.com>

On Thu, Mar 25, 2010 at 6:33 PM, Adam Olsen <rhamph at gmail.com> wrote:
> IMO, we should go for the simplest of all usage: a NaN singleton,
> shared by float and Decimal, compares equal to itself, unsortable, no
> payload.

I don't think disposing of payloads is much of an option for Decimal:
it would break hundreds (literally!) of the decimal testcases, and
would mean that the decimal module would no longer comply with the
standard it's supposed to be based on.

Note that in contrast we're free to alter the meaning of ==, <=,
comparisons for the decimal module, since those aren't covered by the
standard anyway;  someone looking for standards compliance can limit
themselves to using Decimal.compare.

I much prefer Nick's proposal (in the python-dev bit of this thread:
this thread seems to keep moving between python-ideas and python-dev
:).

> IEEE 754 doesn't say NaN should compare false; it says it should
> compare "unordered".

Indeed it does.

> ?We can't do that as we're not using a 4-way
> comparison (which wouldn't generalize to complex anyway), so we have
> to make up our own solution.

Not necessarily:  the standard already gives solutions to the problem
of mapping a 4-way comparison to {true, false}.  It defines (in
section 5.11) a number of predicates that map the four possible
relations (LT, GT, EQ, UN) to true and false.  Amongst those
predicates are 'compareQuietEqual', which maps EQ to True and LT, GT,
UN to False (and hence gives nan == nan --> False), and
'compareSignalingEqual', which does the same but also raises a
floating-point exception for UN.  It doesn't say anything about how
those predicates should be spelled in any particular language, though;
 currently, Python's == corresponds to compareQuietEqual.

Unfortunately neither of those predicates gets us round the reflexivity problem.

Mark


From greg.ewing at canterbury.ac.nz  Fri Mar 26 02:25:06 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 26 Mar 2010 14:25:06 +1300
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <20100325114800.072bfa1a@o>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<AB3214E2-46DC-44FD-8545-6C3FF08FA88D@gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<20100325114800.072bfa1a@o>
Message-ID: <4BAC0CF2.4090600@canterbury.ac.nz>

spir ? wrote:

> (Else there should be a distinction between equality assignment and identity assignemt?
> 	b = a	# ==> a is b and a == b
> 	b := a	# ==> a is b and possibly a == b

Eiffel's position on this seems to be that there should
be no distinction -- a copy of a value should always
compare equal to the original value, regardless of type.

Eiffel even seems to extend this to conversions, so that
if you convert an int to a float, the resulting float should
compare equal to the original int, even if some precision
was lost in the conversion.

(Incidentally, that's one principle we would be choosing
*not* to follow if we decide to compare floats and Decimals
based on their exact values.)

-- 
Greg


From greg.ewing at canterbury.ac.nz  Fri Mar 26 03:01:46 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 26 Mar 2010 15:01:46 +1300
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <83DAEA2C-05CF-45EC-BCE7-0B30490DDEC1@masklinn.net>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<AB3214E2-46DC-44FD-8545-6C3FF08FA88D@gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<83DAEA2C-05CF-45EC-BCE7-0B30490DDEC1@masklinn.net>
Message-ID: <4BAC158A.80601@canterbury.ac.nz>

Masklinn wrote:

> Excuse me but NaNs propagate right? So we'd have not only
> float('nan') == float('nan'), but also float('nan') = float('nan') + 1
> right?

The fact that one of the operands was a NaN propagates,
but it doesn't have to be that particular NaN object.
So NaN + 1 could create a new NaN object to return as
its result.

> [Meyer:]
>> It is rather dangerous indeed to depart from the fundamental laws of mathematics. 
 >
> The very idea of IEEE-754 floats are a
> a definite departure from the fundamental laws of mathematics,

They're a departure from the laws of *real numbers*,
but Meyer is using the word "mathematics" in a much
broader sense here -- meaning any formal system
that you can reason about rigorously.

If your formal system has a notion of equality but
includes values that don't compare equal to themselves,
it seriously screws up your ability to reason about it.

For Meyer, this is a *big* problem, because making it
possible to reason rigorously about programs is something
that Eiffel gets really anal about^H^H^H^H^H^H^H^H^H^H
^H^H^H^H^H^H one of the underpinnings of Eiffel's design.

The more I think about it, the more I think that the
only reason for needing NaNs in the first place is if
you don't have, or don't want to use, an exception
mechanism.

As Meyer points out, treating NaN == NaN as false is
not fundamentally any more correct than treating it
as true. One interpretation or the other might be
appropriate in a particular algorithm, but this needs
to be considered on a case-by-case basis.

So the *pythonic* way to handle NaNs is not to have
them at all, but to raise an exception whenever an
operation would produce a NaN. Code that knows what
to do can catch the exception and proceed appropriately.

Another possibility is to extend the boolean type with
a NaB that propagates in the same way as a NaN. But
something still has to give somewhere, because what
happens when you do this?

   x = float('nan')
   y = float('nan')
   if x == y:
     ...
   else:
     ...

If NaN == NaN is a NaB, then neither branch of the
'if' is appopriate, so the only correct thing to do
would be to raise an exception when trying to branch
on a NaB.

So we have three correctness-preserving possibilites:

1) Always raise an exception as soon as a NaN is
    produced.

2) Propagate NaNs through arithmetic but raise an
    exception when attempting to compare them.

3) Return a NaB when comparing NaNs, and raise an
    exception when attempting to branch on a NaB.

-- 
Greg


From rhamph at gmail.com  Fri Mar 26 03:26:28 2010
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 25 Mar 2010 20:26:28 -0600
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <4BAC158A.80601@canterbury.ac.nz>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<83DAEA2C-05CF-45EC-BCE7-0B30490DDEC1@masklinn.net>
	<4BAC158A.80601@canterbury.ac.nz>
Message-ID: <aac2c7cb1003251926o61e53d05k48dc6475feabf5d3@mail.gmail.com>

On Thu, Mar 25, 2010 at 20:01, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> So we have three correctness-preserving possibilites:
>
> 1) Always raise an exception as soon as a NaN is
> ? produced.
>
> 2) Propagate NaNs through arithmetic but raise an
> ? exception when attempting to compare them.

This forces you to explicitly compare them when desired, but it's even
simpler than I realized (and doesn't require new functions).

def func(a, b):
   if not isnan(a) and not isnan(b) and a == b:
       return 1.0
   return math.sin(a*b)**(a-b)

All it requires is you check for NaN before your normal comparison.
It's even backwards compatible with Python 2.6!

If that example is a little verbose for you, just make isnan take
multiple arguments and return true if any are NaN.


> 3) Return a NaB when comparing NaNs, and raise an
> ? exception when attempting to branch on a NaB.


-- 
Adam Olsen, aka Rhamphoryncus


From denis.spir at gmail.com  Fri Mar 26 11:07:39 2010
From: denis.spir at gmail.com (spir =?UTF-8?B?4pij?=)
Date: Fri, 26 Mar 2010 11:07:39 +0100
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <4BAC0CF2.4090600@canterbury.ac.nz>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<AB3214E2-46DC-44FD-8545-6C3FF08FA88D@gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<20100325114800.072bfa1a@o> <4BAC0CF2.4090600@canterbury.ac.nz>
Message-ID: <20100326110739.464fa248@o>

On Fri, 26 Mar 2010 14:25:06 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Eiffel's position on this seems to be that there should
> be no distinction -- a copy of a value should always
> compare equal to the original value, regardless of type.

Exactly, as I understand Eiffel. And, actually, I tend to support this point of view.

> Eiffel even seems to extend this to conversions, so that
> if you convert an int to a float, the resulting float should
> compare equal to the original int, even if some precision
> was lost in the conversion.

Yes, there is a post-condition equivalent to "assert(newval==val)". That's why (from this point of view):
>>> int(1.2)
1
>>> int(1.7)
1
should not even be possible (--> exception). The programmer has to tell the language about losing information, and thus losing equality, using eg round().
[The same indeed as when converting a huge int to float.]

> (Incidentally, that's one principle we would be choosing
> *not* to follow if we decide to compare floats and Decimals
> based on their exact values.)

What do you mean, exactly?
There may be a reference type, then the condition is Ref(val)==Ref(newval).
Or both type are referent (eg between int and float there may be loss of info in both directions).


Denis
________________________________

vit esse estrany ?

spir.wikidot.com


From denis.spir at gmail.com  Sat Mar 27 10:30:03 2010
From: denis.spir at gmail.com (spir =?UTF-8?B?4pij?=)
Date: Sat, 27 Mar 2010 10:30:03 +0100
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <4BAC158A.80601@canterbury.ac.nz>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<AB3214E2-46DC-44FD-8545-6C3FF08FA88D@gmail.com>
	<d38f5331003241530r1c4e60ceo6979d31b67634f98@mail.gmail.com>
	<5c6f2a5d1003241531m54e73c8dhd844369fe8cf155c@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<83DAEA2C-05CF-45EC-BCE7-0B30490DDEC1@masklinn.net>
	<4BAC158A.80601@canterbury.ac.nz>
Message-ID: <20100327103003.24db568b@o>

On Fri, 26 Mar 2010 15:01:46 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> The more I think about it, the more I think that the
> only reason for needing NaNs in the first place is if
> you don't have, or don't want to use, an exception
> mechanism.
> 
> As Meyer points out, treating NaN == NaN as false is
> not fundamentally any more correct than treating it
> as true. One interpretation or the other might be
> appropriate in a particular algorithm, but this needs
> to be considered on a case-by-case basis.

[In a language or a given case where yielding a NaN does not raise an exception from scratch.]

It seems to me there are various comparison cases (esp. in an OO language, case 2):
-1- Comparing (any) Nan to non_NaN should return False.
	a = <expr yielding a NaN>
	b = 1
	a == b 	# ==> False
-2- Comparing a given NaN with itself should return True.
	a = <expr yielding a NaN>
	b = a
	a == b 	# ==> True
-3- Comparing various NaNs is the undefined case. Actually, I think it's the only case where a theoretical NaB makes sense. In my opinion, as Greg says, the python way would here be raising an exception. But only in that case.


Denis
________________________________

vit esse estrany ?

spir.wikidot.com


From qrczak at knm.org.pl  Sat Mar 27 11:21:17 2010
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Sat, 27 Mar 2010 11:21:17 +0100
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <4BAC0CF2.4090600@canterbury.ac.nz>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com> 
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com> 
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com> 
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com> 
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com> 
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com> 
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com> 
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com> 
	<20100325114800.072bfa1a@o> <4BAC0CF2.4090600@canterbury.ac.nz>
Message-ID: <3f4107911003270321u43638252tc59506a22236d062@mail.gmail.com>

2010/3/26 Greg Ewing <greg.ewing at canterbury.ac.nz>:

> Eiffel even seems to extend this to conversions, so that
> if you convert an int to a float, the resulting float should
> compare equal to the original int, even if some precision
> was lost in the conversion.

Such equality is not transitive (unless 1000000000 and 1000000001 are
equal as ints which would be nonsense).

The original problem with NaN is a consequence of an unfortunate
decision to unify numeric equality with object equivalence. If they
were distinguished, their behavior would be obvious:
  NaN != NaN
  NaN eq NaN
  0.0 == -0.0
  0.0 ne -0.0
  42 == 42.0
  42 ne 42.0
Hash tables would use object equivalence of course.

If you have a type for a time which includes the local time and the
time zone, and < > <= >= compare which time is earlier, then numeric
== should be true for times denoting the same time through different
time zones, but they should not be equivalent.

Lisp and Scheme distinguish these relations.

-- 
Marcin Kowalczyk


From dickinsm at gmail.com  Sat Mar 27 13:28:20 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 27 Mar 2010 12:28:20 +0000
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <3f4107911003270321u43638252tc59506a22236d062@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<20100325114800.072bfa1a@o> <4BAC0CF2.4090600@canterbury.ac.nz>
	<3f4107911003270321u43638252tc59506a22236d062@mail.gmail.com>
Message-ID: <5c6f2a5d1003270528p38bf1889m2583e2ebf1ea7b6f@mail.gmail.com>

On Sat, Mar 27, 2010 at 10:21 AM, Marcin 'Qrczak' Kowalczyk
<qrczak at knm.org.pl> wrote:
> The original problem with NaN is a consequence of an unfortunate
> decision to unify numeric equality with object equivalence. If they
> were distinguished, their behavior would be obvious:

Agreed, except that it's not clear to me that this was actually an
unfortunate decision.  The results in this context are unfortunate,
yes, but it could well be that distinguishing numeric equality and
object equivalence would add unacceptable complexity to the language
for those trying to learn it.  Questions about 'is' versus '==' are
especially common on the mailing lists, and adding a third equivalence
relation wouldn't help newcomers.

Mark


From dickinsm at gmail.com  Sat Mar 27 13:39:13 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 27 Mar 2010 12:39:13 +0000
Subject: [Python-ideas] [Python-Dev] Why is nan != nan?
In-Reply-To: <201003271105.28852.steve@pearwood.info>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<201003261157.39250.steve@pearwood.info>
	<20100326011901.6F4133A4080@sparrow.telecommunity.com>
	<201003271105.28852.steve@pearwood.info>
Message-ID: <5c6f2a5d1003270539mc6cb47fs6cf8222a077ccfe1@mail.gmail.com>

[Moving back to python-ideas]

On Sat, Mar 27, 2010 at 12:05 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Certainly not. That defeats the whole purpose of NANs. I wish floating
> point calculations in Python would return NANs rather than raise the
> exceptions they do now.

I'd prefer the opposite:  that arithmetic operations and math
functions raise rather than produce nans;  this seems friendlier for
users who don't want to know or care about infinities and nans.  But I
agree that there are uses for nonstop mode.

<python-ideas-territory>

How about default behaviour like the above, with a "from __options__
import non-stop-arithmetic" for people who want 1./0. to give infinity
and sqrt(-1) to give nan?  In order words, default behaviour roughly
corresponds to the IEEE 754 spec with the 'invalid operation',
'overflow' and 'divide-by-zero' operations trapped (exactly as the
default context in the decimal module does currently), and the
non-stop behaviour corresponds to IEEE 754 with none of those
operations trapped and returning default values.

A more elaborate proposal would allow individual control over the
three signals above.  (Full control over the 'underflow' and 'inexact'
signals isn't really feasible given C's poor support for these
things.)

</python-ideas-territory>.

Mark


From dickinsm at gmail.com  Sat Mar 27 14:22:54 2010
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 27 Mar 2010 13:22:54 +0000
Subject: [Python-ideas] [Python-Dev] Why is nan != nan?
In-Reply-To: <5c6f2a5d1003270539mc6cb47fs6cf8222a077ccfe1@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<201003261157.39250.steve@pearwood.info>
	<20100326011901.6F4133A4080@sparrow.telecommunity.com>
	<201003271105.28852.steve@pearwood.info>
	<5c6f2a5d1003270539mc6cb47fs6cf8222a077ccfe1@mail.gmail.com>
Message-ID: <5c6f2a5d1003270622w622d172el9f91bd8bf8632aa7@mail.gmail.com>

On Sat, Mar 27, 2010 at 12:39 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> <python-ideas-territory>
>
> How about default behaviour like the above, with a "from __options__
> import non-stop-arithmetic" for people who want 1./0. to give infinity
> and sqrt(-1) to give nan?

Hmm.  That would be a horrible way to control the option.  But the
idea's the same:  give some  way to enable non-stop arithmetic.

Some sort of context manager might make sense:

with non_stop_arithmetic:
    result = calculation()
if isnan(result):
    ...


Mark


From stephen at xemacs.org  Sat Mar 27 16:46:57 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 28 Mar 2010 00:46:57 +0900
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <3f4107911003270321u43638252tc59506a22236d062@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<d38f5331003241536n60a14757v20983a9ec7936a2d@mail.gmail.com>
	<5c6f2a5d1003241547jefbc810off2db2ba07e4d2e0@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<20100325114800.072bfa1a@o> <4BAC0CF2.4090600@canterbury.ac.nz>
	<3f4107911003270321u43638252tc59506a22236d062@mail.gmail.com>
Message-ID: <87k4sxg426.fsf@uwakimon.sk.tsukuba.ac.jp>

Marcin 'Qrczak' Kowalczyk writes:

 > The original problem with NaN is a consequence of an unfortunate
 > decision to unify numeric equality with object equivalence. If they
 > were distinguished, their behavior would be obvious:
 >   NaN != NaN
 >   NaN eq NaN

What do you mean by that?  NaN is a class, not an instance!

 >   0.0 == -0.0
 >   0.0 ne -0.0
 >   42 == 42.0
 >   42 ne 42.0

As for the other examples, I can only sigh, 'Ah, the joys of "general
abstract nonsense".'[1]  I have some (abstract) sympathy for Mark's
proposal for a with_nonstop_arithmetic context manager, but isn't it
really a YAGNI for the Python language?  (As opposed to high
performance modules like numpy.)

Footnotes: 
[1]  I refer to Bourbaki's(?) name for category theory.



From bruce at leapyear.org  Sun Mar 28 00:47:34 2010
From: bruce at leapyear.org (Bruce Leban)
Date: Sat, 27 Mar 2010 16:47:34 -0700
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <87k4sxg426.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com> 
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com> 
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com> 
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com> 
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com> 
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com> 
	<20100325114800.072bfa1a@o> <4BAC0CF2.4090600@canterbury.ac.nz> 
	<3f4107911003270321u43638252tc59506a22236d062@mail.gmail.com> 
	<87k4sxg426.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <cf5b87741003271647s3a8aac44n23fa48360b7de5a7@mail.gmail.com>

I'm missing something from this discussion. Is this an academic exercise or
are there scenarios that don't work well with things as they are? It's hard
to determine if a suggested change solve the problem without knowing what
the problem is.

Here's a problem: I find it annoying that in some (other) languages there's
no easy way to trap integer overflow. It's easy to write code that's wrong
accidentally. But changing the language definition to fail when an overflow
happens would break some valid programs. In python, results that return nan
or inf aren't exactly silent failures but they are quiet. (Personally, I
would have made them non-hashable so you couldn't accidentally stick one in
a dict but that's another problem.)

Anyway, of silent/quiet failure is the issue, then what we need is a way to
make the failure not silent, e.g., wrap it in a try/except block like this
that says, convert any nans to NanErrors:

    try with NanError:
      blah
    except NanError:
      blah

Inside this scope, it might be useful to run code that does allow nans,
which means we'd also need something like:

    without NanError:
      blah

This all seems a bit too magic in how it would work. This is along the line
of Mark Dickinson's suggestion, but the default behavior should stay the
same as it is now.

--- Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100327/72c6e1de/attachment.html>

From stephen at xemacs.org  Sun Mar 28 10:04:08 2010
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 28 Mar 2010 17:04:08 +0900
Subject: [Python-ideas] Why is nan != nan?
In-Reply-To: <cf5b87741003271647s3a8aac44n23fa48360b7de5a7@mail.gmail.com>
References: <5c6f2a5d1003241226y198bc036le862cbd0a4e886e4@mail.gmail.com>
	<d38f5331003241552qf5afd9fsda4cee6f6aa3ce1e@mail.gmail.com>
	<5c6f2a5d1003241602h6567ad30p80f3cba70483a32@mail.gmail.com>
	<d38f5331003241611n777587beq99890761849780f3@mail.gmail.com>
	<5c6f2a5d1003241631m3c6af7e8p93273583cb322b73@mail.gmail.com>
	<d38f5331003242350t9953385if54e72d80219dd97@mail.gmail.com>
	<20100325114800.072bfa1a@o> <4BAC0CF2.4090600@canterbury.ac.nz>
	<3f4107911003270321u43638252tc59506a22236d062@mail.gmail.com>
	<87k4sxg426.fsf@uwakimon.sk.tsukuba.ac.jp>
	<cf5b87741003271647s3a8aac44n23fa48360b7de5a7@mail.gmail.com>
Message-ID: <87iq8gg9dz.fsf@uwakimon.sk.tsukuba.ac.jp>

Bruce Leban writes:

 > (Personally, I would have made them non-hashable so you couldn't
 > accidentally stick one in a dict but that's another problem.)

No, it's not.  AIUI, that is one of the main issues that's being
discussed in this thread.  Most people seem reasonably satisfied with
other behavior, so the discussion bounces back and forth between
whether NaNs should be hashable (and, perhaps more often, whether they
should all be equal or not), and how any change in identity or
equality behavior might affect computations in existing programs.