floats coerced to string with "{:f}".format() ?
Hi everyone,
I've looked in the mailing list archives and with the googles, but haven't yet found any hints with this question...
I have a float field in a NumPy record that looks like it's being substituted as a string in the Python "{:f}".format() minilanguage, thus throwing an error:
In [1]: tmp = np.rec.array([('XYZZ', 2001123, 23.82396)], dtype=np.dtype([('sta', 'S6'), ('ondate', '<i8'), ('lat', '<f4')]))[0]
In [2]: type(tmp) Out[3]: numpy.core.records.record
In [3]: tmp Out[3]: ('XYZZ', 2001123, 23.823917388916016)
In [4]: tmp.sta, tmp.ondate, tmp.lat Out[4]: ('XYZZ', 2001123, 23.823917)
# strings and integers work In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp) Out[5]: 'XYZZ 2001123'
# "lat" is a float, but it seems to be coerced to a string first, and failing In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)  ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput312bff8066cfde8> in <module>() > 1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6f}'.format(tmp)
ValueError: Unknown format code 'f' for object of type 'str'
# string formatting for doesn't fail In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6s}'.format(tmp) Out[7]: 'XYZZ 2001123 23.82'
This also fails:
In [7]: "{:f}".format(np.array(3.2, dtype='f4'))  ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput31433119128e3e6> in <module>() > 1 "{:f}".format(np.array(3.2, dtype='f4'))
ValueError: Unknown format code 'f' for object of type 'str'
Does anyone understand what's happening?
Thanks for your help.
Best, Jon
You can treat a record in a record array like a tuple or a dictionary when it comes to formatting. So, either refer to the index element you want formatted as a float, or refer to it by name (in the formatting language). By just doing {:f}, you are just grabbing the first one, which is "XXYYZZ" and trying to format that. But remember, you can only do this to a single record at a time, not the entire record array at once.
Regular numpy arrays can not be formatted in this manner, hence your other attempt failures.
Cheers! Ben Root
On Thu, Jun 6, 2013 at 3:48 PM, Maccarthy, Jonathan K jkmacc@lanl.govwrote:
Hi everyone,
I've looked in the mailing list archives and with the googles, but haven't yet found any hints with this question...
I have a float field in a NumPy record that looks like it's being substituted as a string in the Python "{:f}".format() minilanguage, thus throwing an error:
In [1]: tmp = np.rec.array([('XYZZ', 2001123, 23.82396)], dtype=np.dtype([('sta', 'S6'), ('ondate', '<i8'), ('lat', '<f4')]))[0]
In [2]: type(tmp) Out[3]: numpy.core.records.record
In [3]: tmp Out[3]: ('XYZZ', 2001123, 23.823917388916016)
In [4]: tmp.sta, tmp.ondate, tmp.lat Out[4]: ('XYZZ', 2001123, 23.823917)
# strings and integers work In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp) Out[5]: 'XYZZ 2001123'
# "lat" is a float, but it seems to be coerced to a string first, and failing In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput312bff8066cfde8> in <module>() > 1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6f}'.format(tmp)
ValueError: Unknown format code 'f' for object of type 'str'
# string formatting for doesn't fail In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6s}'.format(tmp) Out[7]: 'XYZZ 2001123 23.82'
This also fails:
In [7]: "{:f}".format(np.array(3.2, dtype='f4'))
ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput31433119128e3e6> in <module>() > 1 "{:f}".format(np.array(3.2, dtype='f4'))
ValueError: Unknown format code 'f' for object of type 'str'
Does anyone understand what's happening?
Thanks for your help.
Best, Jon _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Thu, Jun 6, 2013 at 8:48 PM, Maccarthy, Jonathan K jkmacc@lanl.gov wrote:
Hi everyone,
I've looked in the mailing list archives and with the googles, but haven't yet found any hints with this question...
I have a float field in a NumPy record that looks like it's being substituted as a string in the Python "{:f}".format() minilanguage, thus throwing an error:
In [1]: tmp = np.rec.array([('XYZZ', 2001123, 23.82396)], dtype=np.dtype([('sta', 'S6'), ('ondate', '<i8'), ('lat', '<f4')]))[0]
In [2]: type(tmp) Out[3]: numpy.core.records.record
In [3]: tmp Out[3]: ('XYZZ', 2001123, 23.823917388916016)
In [4]: tmp.sta, tmp.ondate, tmp.lat Out[4]: ('XYZZ', 2001123, 23.823917)
# strings and integers work In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp) Out[5]: 'XYZZ 2001123'
# "lat" is a float, but it seems to be coerced to a string first, and failing In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput312bff8066cfde8> in <module>() > 1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6f}'.format(tmp)
ValueError: Unknown format code 'f' for object of type 'str'
# string formatting for doesn't fail In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6s}'.format(tmp) Out[7]: 'XYZZ 2001123 23.82'
This also fails:
In [7]: "{:f}".format(np.array(3.2, dtype='f4'))
ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput31433119128e3e6> in <module>() > 1 "{:f}".format(np.array(3.2, dtype='f4'))
ValueError: Unknown format code 'f' for object of type 'str'
Does anyone understand what's happening?
numpy.ndarray does not implement the __format__() method. Thus, str.format() method falls back to object.__format__(). This is the exception that object.__format__() raises. Why it says "object of type 'str'" is not clear to me. Similarly, numpy.float32 does not implement the __format__() method. The string scalar type and the native integer scalar type (I assume you are on a 64bit platform, so Python ints are 64bit to match your 'i8' field) inherit from the corresponding native Python types, so they inherit their __format__() methods.
 Robert Kern
Ah, so np.int64 and np.str inherit the native Python __format__(), but np.float32/64 doesn't get __builtin__.float.__format__(). That's not intuitive, but I see now why this works:
In [8]: '{:6.6s} {:8d} {:11.6f}'.format(tmp.sta, tmp.ondate, float(tmp.lat)) Out[8]: 'XYZZ 2001123 23.820000'
Thanks!
Jon
On Jun 6, 2013, at 1:56 PM, Robert Kern wrote:
On Thu, Jun 6, 2013 at 8:48 PM, Maccarthy, Jonathan K jkmacc@lanl.gov wrote:
Hi everyone,
I've looked in the mailing list archives and with the googles, but haven't yet found any hints with this question...
I have a float field in a NumPy record that looks like it's being substituted as a string in the Python "{:f}".format() minilanguage, thus throwing an error:
In [1]: tmp = np.rec.array([('XYZZ', 2001123, 23.82396)], dtype=np.dtype([('sta', 'S6'), ('ondate', '<i8'), ('lat', '<f4')]))[0]
In [2]: type(tmp) Out[3]: numpy.core.records.record
In [3]: tmp Out[3]: ('XYZZ', 2001123, 23.823917388916016)
In [4]: tmp.sta, tmp.ondate, tmp.lat Out[4]: ('XYZZ', 2001123, 23.823917)
# strings and integers work In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp) Out[5]: 'XYZZ 2001123'
# "lat" is a float, but it seems to be coerced to a string first, and failing In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput312bff8066cfde8> in <module>() > 1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6f}'.format(tmp)
ValueError: Unknown format code 'f' for object of type 'str'
# string formatting for doesn't fail In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:>11.6s}'.format(tmp) Out[7]: 'XYZZ 2001123 23.82'
This also fails:
In [7]: "{:f}".format(np.array(3.2, dtype='f4'))
ValueError Traceback (most recent call last) /Users/jkmacc/<ipythoninput31433119128e3e6> in <module>() > 1 "{:f}".format(np.array(3.2, dtype='f4'))
ValueError: Unknown format code 'f' for object of type 'str'
Does anyone understand what's happening?
numpy.ndarray does not implement the __format__() method. Thus, str.format() method falls back to object.__format__(). This is the exception that object.__format__() raises. Why it says "object of type 'str'" is not clear to me. Similarly, numpy.float32 does not implement the __format__() method. The string scalar type and the native integer scalar type (I assume you are on a 64bit platform, so Python ints are 64bit to match your 'i8' field) inherit from the corresponding native Python types, so they inherit their __format__() methods.
 Robert Kern _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Thu, Jun 6, 2013 at 9:18 PM, Maccarthy, Jonathan K jkmacc@lanl.gov wrote:
Ah, so np.int64 and np.str inherit the native Python __format__(), but np.float32/64 doesn't get __builtin__.float.__format__(). That's not intuitive, but I see now why this works:
In [8]: '{:6.6s} {:8d} {:11.6f}'.format(tmp.sta, tmp.ondate, float(tmp.lat)) Out[8]: 'XYZZ 2001123 23.820000'
np.float64 works because it inherits from the Python float type (Python floats are 64bit floats). np.float32 doesn't inherit from the Python float type because it can't; they don't represent the same kind of data, so their memory layouts at the C level cannot coincide. Since you are on a 64bit platform, np.int64 represents the same kind of integer as the Python int type, so it can subclass, but an np.int32 couldn't.
It's not necessarily intuitive, but it's the best we can do under the constraints. The only thing more intuitive would be to disallow subclassing from the Python builtin types entirely, but that's *really* annoying.
 Robert Kern
I'm really showing my ignorance now, I think; so, the oldstyle "fmt % (tuple)" must use a different mechanism, and perhaps that's why np.savetxt never choked on a float32 for me before (yes, I am on a 64bit system).
In [8]: type(tmp.lat) Out[8]: numpy.float32
In [9]: '%6s %8i %11.6f' % (tmp.sta, tmp.ondate, tmp.lat) Out[9]: ' XYZZ 2001123 23.820000'
Thanks again for your patience.
On Jun 6, 2013, at 2:26 PM, Robert Kern wrote:
np.float64 works because it inherits from the Python float type (Python floats are 64bit floats). np.float32 doesn't inherit from the Python float type because it can't; they don't represent the same kind of data, so their memory layouts at the C level cannot coincide. Since you are on a 64bit platform, np.int64 represents the same kind of integer as the Python int type, so it can subclass, but an np.int32 couldn't.
It's not necessarily intuitive, but it's the best we can do under the constraints. The only thing more intuitive would be to disallow subclassing from the Python builtin types entirely, but that's *really* annoying.
 Robert Kern _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Thu, Jun 6, 2013 at 9:50 PM, Maccarthy, Jonathan K jkmacc@lanl.gov wrote:
I'm really showing my ignorance now, I think; so, the oldstyle "fmt % (tuple)" must use a different mechanism, and perhaps that's why np.savetxt never choked on a float32 for me before (yes, I am on a 64bit system).
In [8]: type(tmp.lat) Out[8]: numpy.float32
In [9]: '%6s %8i %11.6f' % (tmp.sta, tmp.ondate, tmp.lat) Out[9]: ' XYZZ 2001123 23.820000'
Thanks again for your patience.
Yes, the mechanism is quite different. With %f, str.__mod__() is in control. When it is handling the %f code, it is expecting a Python float object *or* one that can be converted into one via float(). All of the numpy floating and integer types can be converted to a Python float via float().
With str.format(), the control is inverted. The value being represented is asked via its __format__() method to handle the 'f' format code.
 Robert Kern
participants (3)

Benjamin Root

Maccarthy, Jonathan K

Robert Kern