[ python-Bugs-1417699 ] float/atof have become locale aware

[SourceForge.net]

Bugs item #1417699, was opened at 2006-01-29 01:04
Message generated for change (Comment added) made by gustavo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1417699&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 7
Submitted By: Bernhard Herzog (bernhard)
Assigned to: Nobody/Anonymous (nobody)
Summary: float/atof have become locale aware

Initial Comment:
The builtin float and the function string.atof have
become locale aware in Python 2.4:

Python 2.4.2 (#1, Nov 29 2005, 16:07:55) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import locale
>>> import string
>>> locale.setlocale(locale.LC_ALL, "de_DE")
'de_DE'
>>> float("1,5")
1.5
>>> string.atof("1,5")
1.5


This doesn't match what's specified in pep 331:

        - float() and str() stay locale-unaware.

It also doesn't match the documentation for atof:

  Convert a string to a floating point number.
  The string must have the standard syntax for a
  floating point literal in Python, optionally
  preceded by a sign ("+" or "-"). Note that this
  behaves identical to the built-in function float()
  when passed a string.

The documentation for float() doesn't state which
format is accepted by float(), though.

The reason for this behavior is ultimately, that
PyOS_ascii_strtod accepts the locale specific
convention in addition to the "C"-locale/python
convention, even though the comment in the code
preceding its definition states:

  This function behaves like the standard strtod() 
  function does in the C locale.




----------------------------------------------------------------------

Comment By: Gustavo J. A. M. Carneiro (gustavo)
Date: 2006-05-13 12:59

Message:
Logged In: YES 
user_id=908

I'm quite busy at this moment, but I'll send a patch soon,
promise.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-05-12 15:14

Message:
Logged In: YES 
user_id=21627

Unassigning myself - I don't plan to work on this anytime soon.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-05-08 19:00

Message:
Logged In: YES 
user_id=849994

I grepped around and found PyOS_ascii_strtod in the
following places:

* stropmodule::atof
* cPickle::load_float (the writing of locale dependent
floats was fixed recently)
* floatobject::PyFloat_FromString
* complexobject::complex_subtype_from_string

In my opinion, all these should not tolerate localized
floats, so Gustavo, please ask the original author how to
achieve this.

----------------------------------------------------------------------

Comment By: Gustavo J. A. M. Carneiro (gustavo)
Date: 2006-05-04 10:47

Message:
Logged In: YES 
user_id=908

My comment is, is "PyOS_ascii_strtod accepting both forms"
something that is harmful and should be fixed?

I didn't exactly write the PyOS_ascii_strtod code; I only
integrated it with Python.  But if we really need to fix
this, I'll try to figure out how to make it not accept
floating points in locale aware format; if necessary I'll
ask help from  the original author of the code, Alexander
Larsson.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-05-04 05:42

Message:
Logged In: YES 
user_id=21627

It's a bug. As bernhard says, it originates from
PyOS_ascii_strtod accepting both forms, which in turn
happens because it just leaves the locale-specific decimal
point in the string, only replacing the . with the
locale-specific point. Then, the platform's strtod will
happily accept the locale-specific version. 

I'd like Gustavo Carneiro to comment.

----------------------------------------------------------------------

Comment By: Georg Brandl (birkenfeld)
Date: 2006-02-17 12:19

Message:
Logged In: YES 
user_id=1188172

Martin, you checked in the patch which is mentioned in PEP
331. Is this correct behavior?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1417699&group_id=5470