Compact repr operator (i.e., __short_repr__)
I was curious if what kind of interest there would be in adding an overridable method to types to allow getting back a repr, but a predictably short one. The motivation is that there are many cases where producing the full, round-trippable repr is possible, but would take up significant resources or be too huge to be useful for human consumption. The built-in [reprlib module][1] certainly shows there's an interest for many built-in types, and I think arguably that same functionality is desirable for user types. As mentioned in the subject a possible API could be: class ValueObject(object): def __init__(self, content): self.content = content def __repr__(self): return '%s(%r)' % (self.__class__.__name__, self.content) def __short_repr__(self, size=None, depth=None): # TODO: should interpretation of size/depth be casual or strict? if size and len(self.content) > size: short_content = self.content[:size] + '...' else: pass # TODO: could just return normal repr possibly return '<%s content=%r)' % (self.__class__.__name__, short_content) Without something like this, there's no way to ask an object if it's repr is of manageable length, or how its repr could be meaningfully shortened. Usually I just chop out the middle and add an ellipsis, but as for the time spent generating that middle, I'll never get those cycles back. Anyways, thanks for your attention, and happy Monday to all! Mahmoud https://github.com/mahmoud https://twitter.com/mhashemi http://sedimental.org [1]: https://docs.python.org/2/library/functions.html#func-repr [2]: https://docs.python.org/3.4/library/reprlib.html
On Mon, Feb 8, 2016 at 3:36 PM, Mahmoud Hashemi
The motivation is that there are many cases where producing the full, round-trippable repr is possible, but would take up significant resources or be too huge to be useful for human consumption.
numpy as realized this, and produced a __repr__ (and __str__) that truncates: In [*26*]: len(arr) Out[*26*]: 100000 In [*27*]: repr(arr) Out[*27*]: 'array([ 0.00000000e+00, 1.00001000e-02, 2.00002000e-02, ...,\n 9.99980000e+02, 9.99990000e+02, 1.00000000e+03])' I"m not sure that a full-sized repr is ever useful, so this seems fine to me. I wonder how often anyone actually counts on eval(repr(obj)) == obj ? In short, I don't see that this would be all that useful. -CHB
The built-in [reprlib module][1] certainly shows there's an interest for many built-in types, and I think arguably that same functionality is desirable for user types.
As mentioned in the subject a possible API could be:
class ValueObject(object): def __init__(self, content): self.content = content
def __repr__(self): return '%s(%r)' % (self.__class__.__name__, self.content)
def __short_repr__(self, size=None, depth=None): # TODO: should interpretation of size/depth be casual or strict? if size and len(self.content) > size: short_content = self.content[:size] + '...' else: pass # TODO: could just return normal repr possibly return '<%s content=%r)' % (self.__class__.__name__, short_content)
Without something like this, there's no way to ask an object if it's repr is of manageable length, or how its repr could be meaningfully shortened. Usually I just chop out the middle and add an ellipsis, but as for the time spent generating that middle, I'll never get those cycles back.
Anyways, thanks for your attention, and happy Monday to all!
Mahmoud https://github.com/mahmoud https://twitter.com/mhashemi http://sedimental.org
[1]: https://docs.python.org/2/library/functions.html#func-repr [2]: https://docs.python.org/3.4/library/reprlib.html
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Feb 8, 2016 at 6:36 PM Mahmoud Hashemi
Without something like this, there's no way to ask an object if it's repr is of manageable length, or how its repr could be meaningfully shortened. Usually I just chop out the middle and add an ellipsis, but as for the time spent generating that middle, I'll never get those cycles back.
With the addition of ``__short_repr__`` would there be a change to the default Python REPL? If not, how would you use the new magic method? If so, then why not just change __repr__ for all the built-ins to truncate automatically? As far as I know, reprs are not part of the backwards compatibility guarantee. (Note, I'm not advocating for changing reprs.) If the usage would be to write a function that checks for the existence of __short_repr__, then why not simply move the implementation of ``if len(s) < maxlen`` etc. to that function?
Roundtrippable reprs are certainly part of Python canon, whether or not
they are universally used (Chris), or guaranteed (Mike).
I could see __short_repr__ (and associated reprlib) being the desired
behavior in some console environments, but I'm not one to say it should be
the default in the main Python REPL.
My use cases are for 1) a web based console/REPL and 2) a configuration
store that maintains a human readable history of past values (i.e., it does
not maintain references to the objects themselves).
But mostly I wanted to kick off the discussion of how to update reprlib
(and pprint) to be more efficient and applicable.
Mahmoud
On Mon, Feb 8, 2016 at 4:28 PM, Michael Selik
On Mon, Feb 8, 2016 at 6:36 PM Mahmoud Hashemi
wrote: Without something like this, there's no way to ask an object if it's repr is of manageable length, or how its repr could be meaningfully shortened. Usually I just chop out the middle and add an ellipsis, but as for the time spent generating that middle, I'll never get those cycles back.
With the addition of ``__short_repr__`` would there be a change to the default Python REPL? If not, how would you use the new magic method? If so, then why not just change __repr__ for all the built-ins to truncate automatically? As far as I know, reprs are not part of the backwards compatibility guarantee. (Note, I'm not advocating for changing reprs.)
If the usage would be to write a function that checks for the existence of __short_repr__, then why not simply move the implementation of ``if len(s) < maxlen`` etc. to that function?
On 9 Feb 2016, at 02:49, Mahmoud Hashemi
wrote: Roundtrippable reprs are certainly part of Python canon, whether or not they are universally used (Chris), or guaranteed (Mike).
They can be part of the canon all they want, but if they’re not universally guaranteed then I don’t know that this is a real problem. It means that the world of Python objects divides into two kinds: first, those with __repr__ return values that can be round tripped and those with __repr__ return values that cannot be round tripped. Given that objects of the second type already exist (I know this for a fact because I have written some quite recently!), it would be an error to assume that the identity 'eval(repr(x)) == x’ holds for arbitrary types. In fact, not only does it not hold for all third-party types, it doesn’t even hold for all built-in types:
x = float(‘nan’) repr(x) ‘nan’ eval(repr(x)) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> NameError: name 'nan' is not defined
I think the reality is that there is no constraint on the representation of arbitrary types to be round-trippable in any way. Again, all custom types have non-round-trippable representations by default, many more eclectic built-in types have non-round-tripppable representations (in addition to NaN, the memoryview object leaps to mind). I can also note the Python documentation on repr:
For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval(), otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object.
If the language doesn’t even try to enforce the idea that representations will be round-trippable, I think there’s just no problem here. Cory
If you want to make the case that default reprs should be
non-roundtrippable in the case that they're too long, that's a fine and
separate discussion. Though I have argued in the past that float('nan')
would be a less surprising/more useful repr.
And that's what it's about, usefulness. It's useful to be able to copy in
and out of the REPL, even for very large reprs. I've done it, my coworkers
and students do it, and you've probably done it as well.
But there are other times (often outside the REPL), where that is not the
case, and being able to address them explicitly, in the vein of reprlib and
pprint -- but better -- would be *useful*. Who hasn't wished that the
built-in defaultdict and OrderedDict were as pprintable or
reprlib.repr-able as dict.
There's plenty of room to improve.
On Tue, Feb 9, 2016 at 12:56 AM, Cory Benfield
On 9 Feb 2016, at 02:49, Mahmoud Hashemi
wrote: Roundtrippable reprs are certainly part of Python canon, whether or not they are universally used (Chris), or guaranteed (Mike).
They can be part of the canon all they want, but if they’re not universally guaranteed then I don’t know that this is a real problem. It means that the world of Python objects divides into two kinds: first, those with __repr__ return values that can be round tripped and those with __repr__ return values that cannot be round tripped.
Given that objects of the second type already exist (I know this for a fact because I have written some quite recently!), it would be an error to assume that the identity 'eval(repr(x)) == x’ holds for arbitrary types. In fact, not only does it not hold for all third-party types, it doesn’t even hold for all built-in types:
x = float(‘nan’) repr(x) ‘nan’ eval(repr(x)) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> NameError: name 'nan' is not defined
I think the reality is that there is no constraint on the representation of arbitrary types to be round-trippable in any way. Again, all custom types have non-round-trippable representations by default, many more eclectic built-in types have non-round-tripppable representations (in addition to NaN, the memoryview object leaps to mind).
I can also note the Python documentation on repr:
For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval(), otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object.
If the language doesn’t even try to enforce the idea that representations will be round-trippable, I think there’s just no problem here.
Cory
The appropriate truncation for any repr that is too long would depend on
the particulars of both the object and the purpose of the output. If
reprlib and pprint were improved for say, defaultdict and OrderedDict, or
more generic things, what would its features be?
I'm guessing you wouldn't just duplicate the functionality of
``pprint(dict(x))``.
On Tue, Feb 9, 2016, 4:09 AM Mahmoud Hashemi
If you want to make the case that default reprs should be non-roundtrippable in the case that they're too long, that's a fine and separate discussion. Though I have argued in the past that float('nan') would be a less surprising/more useful repr.
And that's what it's about, usefulness. It's useful to be able to copy in and out of the REPL, even for very large reprs. I've done it, my coworkers and students do it, and you've probably done it as well.
But there are other times (often outside the REPL), where that is not the case, and being able to address them explicitly, in the vein of reprlib and pprint -- but better -- would be *useful*. Who hasn't wished that the built-in defaultdict and OrderedDict were as pprintable or reprlib.repr-able as dict.
There's plenty of room to improve.
On Tue, Feb 9, 2016 at 12:56 AM, Cory Benfield
wrote: On 9 Feb 2016, at 02:49, Mahmoud Hashemi
wrote: Roundtrippable reprs are certainly part of Python canon, whether or not they are universally used (Chris), or guaranteed (Mike).
They can be part of the canon all they want, but if they’re not universally guaranteed then I don’t know that this is a real problem. It means that the world of Python objects divides into two kinds: first, those with __repr__ return values that can be round tripped and those with __repr__ return values that cannot be round tripped.
Given that objects of the second type already exist (I know this for a fact because I have written some quite recently!), it would be an error to assume that the identity 'eval(repr(x)) == x’ holds for arbitrary types. In fact, not only does it not hold for all third-party types, it doesn’t even hold for all built-in types:
x = float(‘nan’) repr(x) ‘nan’ eval(repr(x)) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> NameError: name 'nan' is not defined
I think the reality is that there is no constraint on the representation of arbitrary types to be round-trippable in any way. Again, all custom types have non-round-trippable representations by default, many more eclectic built-in types have non-round-tripppable representations (in addition to NaN, the memoryview object leaps to mind).
I can also note the Python documentation on repr:
For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval(), otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object.
If the language doesn’t even try to enforce the idea that representations will be round-trippable, I think there’s just no problem here.
Cory
For those examples, the pprint/reprlib behavior probably wouldn't be much
different than dict/list of tuples. I can put my vote in, but I prefer to
defer design specifics to the creators/maintainers of those types. That's
my point.
Right now I feel like repr() is a powerful, but underleveraged function in
the builtin ecosystem. Instead of pprint.pformat(), why not add a keyword
argument, and have repr(obj, pretty=True), and have that call through with
arguments to a more full featured __repr__? Same could go for repr(obj,
length=150, depth=2). These could have defaults set for REPL sessions.
Everyone is doing hacky, incomplete heuristic introspection, from IPython
to the standard library. Instead, let's embrace the complexities of viewing
application state, and allow for formalization of community efforts.
Mahmoud
On Wed, Feb 10, 2016 at 11:18 AM, Michael Selik
The appropriate truncation for any repr that is too long would depend on the particulars of both the object and the purpose of the output. If reprlib and pprint were improved for say, defaultdict and OrderedDict, or more generic things, what would its features be?
I'm guessing you wouldn't just duplicate the functionality of ``pprint(dict(x))``.
On Tue, Feb 9, 2016, 4:09 AM Mahmoud Hashemi
wrote: If you want to make the case that default reprs should be non-roundtrippable in the case that they're too long, that's a fine and separate discussion. Though I have argued in the past that float('nan') would be a less surprising/more useful repr.
And that's what it's about, usefulness. It's useful to be able to copy in and out of the REPL, even for very large reprs. I've done it, my coworkers and students do it, and you've probably done it as well.
But there are other times (often outside the REPL), where that is not the case, and being able to address them explicitly, in the vein of reprlib and pprint -- but better -- would be *useful*. Who hasn't wished that the built-in defaultdict and OrderedDict were as pprintable or reprlib.repr-able as dict.
There's plenty of room to improve.
On Tue, Feb 9, 2016 at 12:56 AM, Cory Benfield
wrote: On 9 Feb 2016, at 02:49, Mahmoud Hashemi
wrote: Roundtrippable reprs are certainly part of Python canon, whether or not they are universally used (Chris), or guaranteed (Mike).
They can be part of the canon all they want, but if they’re not universally guaranteed then I don’t know that this is a real problem. It means that the world of Python objects divides into two kinds: first, those with __repr__ return values that can be round tripped and those with __repr__ return values that cannot be round tripped.
Given that objects of the second type already exist (I know this for a fact because I have written some quite recently!), it would be an error to assume that the identity 'eval(repr(x)) == x’ holds for arbitrary types. In fact, not only does it not hold for all third-party types, it doesn’t even hold for all built-in types:
x = float(‘nan’) repr(x) ‘nan’ eval(repr(x)) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> NameError: name 'nan' is not defined
I think the reality is that there is no constraint on the representation of arbitrary types to be round-trippable in any way. Again, all custom types have non-round-trippable representations by default, many more eclectic built-in types have non-round-tripppable representations (in addition to NaN, the memoryview object leaps to mind).
I can also note the Python documentation on repr:
For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval(), otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object.
If the language doesn’t even try to enforce the idea that representations will be round-trippable, I think there’s just no problem here.
Cory
On 10 Feb 2016, at 19:31, Mahmoud Hashemi
wrote: For those examples, the pprint/reprlib behavior probably wouldn't be much different than dict/list of tuples. I can put my vote in, but I prefer to defer design specifics to the creators/maintainers of those types. That's my point.
Right now I feel like repr() is a powerful, but underleveraged function in the builtin ecosystem. Instead of pprint.pformat(), why not add a keyword argument, and have repr(obj, pretty=True), and have that call through with arguments to a more full featured __repr__? Same could go for repr(obj, length=150, depth=2). These could have defaults set for REPL sessions.
Everyone is doing hacky, incomplete heuristic introspection, from IPython to the standard library. Instead, let's embrace the complexities of viewing application state, and allow for formalization of community efforts.
If we want to embrace the complexities of viewing application state, perhaps we should divorce this conversation from the repr altogether. There’s plenty of applications where “viewing application state” will require more than the repr generally provides: you’ll want insight into private member variables, into the state of composed objects, and potentially into the state of some of those private member variables (e.g. is this lock held at this moment?). This is outside the generally accepted scope of the repr as I understand it: most people don’t provide the repr in this form. If you want the complexity of application state then you probably want some way to dump the graph of objects starting with a specific one, along with their state. That’s totally beyond the scope of the repr, and is closer to what pickle would do for you (though as Andrew points out, pickle is not the right tool for this job either). What I don’t understand is: if you are in the REPL already, what does any of this buy you? The REPL has plenty of powerful tools for introspecting Python objects: specifically, it has the Python programming language! If you want to introspect application state from the REPL, then just grab the object you want and start interrogating its state. dir() works, printing fields works, calling methods works, and all of this is far more powerful than anything you can get from the repr. Cory
Michael: Yes, I have several projects on PyPI, some of which are related
and somewhat popular, but I hardly think that a PyPI package can encompass
the API changes I'm describing. We already have several built-in modules
(reprlib, pprint) and 3rd party modules (IPython Notebook, Django 500
pages, the Werkzeug debugger) that see a great deal of usage, but only
support a few builtinn types and have to hack heuristics for the rest.
Cory: Introspecting application state happens at many levels, one step at a
time. Not only is perfect the enemy of good here, but I really don't think
an omni-solution like the one you're describing is a realistic, Pythonic
goal. Besides, graphs are powerful, but don't go underestimating linear
text.
An enhanced repr allows developer-users to communicate the context of their
usage so that the library authors can programmatically provide and
prioritize information in their types. I've seen (and written) plenty of
reprs that encode useful information in my types. To your example, I might
have:
repr(obj, pretty=True, depth=2, width=120)
Though, as I said before, my primary use case isn't the REPL, and those
arguments should be part of the REPL config (pprint defaults to width=80
for consoles). The point is that repr has room to evolve.
Mahmoud
On Thu, Feb 11, 2016 at 1:00 AM, Cory Benfield
On 10 Feb 2016, at 19:31, Mahmoud Hashemi
wrote: For those examples, the pprint/reprlib behavior probably wouldn't be much different than dict/list of tuples. I can put my vote in, but I prefer to defer design specifics to the creators/maintainers of those types. That's my point.
Right now I feel like repr() is a powerful, but underleveraged function in the builtin ecosystem. Instead of pprint.pformat(), why not add a keyword argument, and have repr(obj, pretty=True), and have that call through with arguments to a more full featured __repr__? Same could go for repr(obj, length=150, depth=2). These could have defaults set for REPL sessions.
Everyone is doing hacky, incomplete heuristic introspection, from IPython to the standard library. Instead, let's embrace the complexities of viewing application state, and allow for formalization of community efforts.
If we want to embrace the complexities of viewing application state, perhaps we should divorce this conversation from the repr altogether.
There’s plenty of applications where “viewing application state” will require more than the repr generally provides: you’ll want insight into private member variables, into the state of composed objects, and potentially into the state of some of those private member variables (e.g. is this lock held at this moment?). This is outside the generally accepted scope of the repr as I understand it: most people don’t provide the repr in this form.
If you want the complexity of application state then you probably want some way to dump the graph of objects starting with a specific one, along with their state. That’s totally beyond the scope of the repr, and is closer to what pickle would do for you (though as Andrew points out, pickle is not the right tool for this job either).
What I don’t understand is: if you are in the REPL already, what does any of this buy you? The REPL has plenty of powerful tools for introspecting Python objects: specifically, it has the Python programming language! If you want to introspect application state from the REPL, then just grab the object you want and start interrogating its state. dir() works, printing fields works, calling methods works, and all of this is far more powerful than anything you can get from the repr.
Cory
On 11 February 2016 at 19:24, Mahmoud Hashemi
Michael: Yes, I have several projects on PyPI, some of which are related and somewhat popular, but I hardly think that a PyPI package can encompass the API changes I'm describing. We already have several built-in modules (reprlib, pprint) and 3rd party modules (IPython Notebook, Django 500 pages, the Werkzeug debugger) that see a great deal of usage, but only support a few builtinn types and have to hack heuristics for the rest.
I generally haven't been following these threads, but this one caught my eye. Making pprint extensible was one of the rationales for adding functools.singledispatch to the standard library. However, as far as I am aware, we haven't had anyone find the time to follow up on that change by actually applying it where it would make sense to do so. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Feb 11, 2016 at 01:24:42AM -0800, Mahmoud Hashemi wrote:
I still think you're selling repr short. It can be much more. One day we might have:
repr(obj, pretty=True, depth=2, width=120)
I'm sympathetic, but I'm not sure that repr() is the right place to be focusing. repr() is too closely tied to the REPL and the existing string formatting codes to start making radical changes. But we do have the pprint library, which (in my opinion) is underpowered and underused. I know that some people think that Python already has too many string conversion functions ( str()/__str__ and repr()/__repr__ ) but for my own use I've occasionally found that it has too few. Some time ago, I started working on a library for continued fractions. I count at least five useful/standard representations for a continued fraction. All of these would be equivalent: ContinuedFraction(1, 2, 3, 4, 5) [1; 2, 3, 4, 5] 1 + 1/(2 + 1/(3 + 1/(4 + 1/5))) 1 1 1 1 1 + ----- ----- ----- ----- 2 + 3 + 4 + 5 1 1 + ----------------- 1 2 + ------------- 1 3 + --------- 1 4 + ----- 5 (Apart from the first, the remaining four are standard notations used in mathematics.) Now obviously I can just add my own string conversion methods to the class, but it would be nice to be able to integrate with the pprint (and maybe reprlib) libraries in some way rather than an ad hoc set of arbitrarily named methods? I must admit that I'm just thinking aloud here, I don't actually have a concrete idea. But maybe my use-case will spark an idea in someone else. -- Steve
On Thu, Feb 11, 2016, at 05:59, Steven D'Aprano wrote:
Some time ago, I started working on a library for continued fractions. I count at least five useful/standard representations for a continued fraction. All of these would be equivalent:
(Apart from the first, the remaining four are standard notations used in mathematics.)
How about 1+\dfrac 1{2+\dfrac 1{3+\dfrac 1{4+\dfrac 1 5}}}? A way to get a "mathematical abstract syntax tree" that can be operated on and transformed into that or any of your notations [1+1/(2+1/(3+1/(4+1/5))) might be the default] for numeric objects (and other things such as matrices and vectors) might be nice.
On 11 Feb 2016, at 09:24, Mahmoud Hashemi
wrote: An enhanced repr allows developer-users to communicate the context of their usage so that the library authors can programmatically provide and prioritize information in their types. I've seen (and written) plenty of reprs that encode useful information in my types. To your example, I might have:
.
To be clear, I absolutely do that as well. In fact, if I’m writing a repr that’s almost always the form I give it. Again, I’m not arguing that the repr isn’t useful. I’m arguing that it *is* useful, and that attempting to extend its utility with flags and complex functionality dilutes its utility. I’m in opposition to trying to shovel more into the repr: to try to extend it with more flexibility and therefore increase the burden on developers to be useful in that scenario. I don’t disagree that the repr *can* be more: I just don’t think it should be. The repr is, like it or not, implicitly unstructured. Because the return value of __repr__ is a string, we cannot easily extend it for utility without pushing that cost out to developers. That is, if we wanted to support your “pretty=True” argument to repr(), all developers would need to start writing __repr__ functions that respond to that flag. This is, IMO, a bad way to obtain the feature you want, because at this point ‘pretty’ becomes constrained by the original author. It also means that some flags may not work if the author has not supported them: ‘pretty’ may do nothing, ‘pretty’ may work but ‘depth’ may be ignored, or ‘width’. What I’ve understood from your mails is that you want is the ability to provide a *structured* debug representation of an object. That is, a function to hang off an object that is formatted according to the preferences of the user, not the developer. I think this is a good thing to have, because structured debug representations allow for more automated processing, as well as additional utility for users: see for example the structlog[0] module, which applies this concept to logging. This is a good and reasonable thing to want, but it’s a different thing to the repr. The repr is fundamentally an unstructured representation of whatever the author believed was useful to describe about the object. I don’t see any reason to remove that or to put pressure on that mechanism to be more than it is. Cory [0]: https://structlog.readthedocs.org/en/stable/
On Feb 9, 2016, at 00:56, Cory Benfield
On 9 Feb 2016, at 02:49, Mahmoud Hashemi
wrote: Roundtrippable reprs are certainly part of Python canon, whether or not they are universally used (Chris), or guaranteed (Mike).
They can be part of the canon all they want, but if they’re not universally guaranteed then I don’t know that this is a real problem. It means that the world of Python objects divides into two kinds: first, those with __repr__ return values that can be round tripped and those with __repr__ return values that cannot be round tripped.
So? One use of round-trippable reprs is to copy them from output and paste them into source code or an interactive session. When doing so, you almost always know that you're dealing with a builtin or third-party type that's round-trippable--and, when you're surprised, it's almost always obvious, because you get something that looks nothing at all like the source equivalent, and that raises a SyntaxError if you try to evaluate it anyway. And even those "almost"s aren't a problem in practice. Sure, a list that recursively contains itself looks misleadingly round-trippable, and will evaluate successfully into the wrong thing. But this rarely comes up in practice--and, if it does, because there's a human being inspecting, debugging, or playing with things, rather than a program, it's easy to deal with. This isn't theoretical--I do this all the time when debugging code, I write my own types to make them easier to debug this way, and it saves me time and hassle. In fact, it's one of the benefits of using Python over some of the other languages I use, where repr or toDebugString or whatever is never useful, instead of being usually useful especially with common types. The only thing the non-universality of round-tripping means is that you can't use repr with eval as a persistence format. Which is a good thing--you *shouldn't* use it as a persistence format, and that would be true even if it did work. But using it as an inspecting/debugging format is not a problem, and breaking that would be a bad idea. In fact, breaking it would make repr nearly pointless. Except for a types that define __repr__ but block __str__ with a TypeError (which is rare, and it's debatable whether those types are even valid), when would you ever use repr otherwise?
On 9 Feb 2016, at 10:13, Andrew Barnert
wrote: So?
One use of round-trippable reprs is to copy them from output and paste them into source code or an interactive session. When doing so, you almost always know that you're dealing with a builtin or third-party type that's round-trippable--and, when you're surprised, it's almost always obvious, because you get something that looks nothing at all like the source equivalent, and that raises a SyntaxError if you try to evaluate it anyway.
And even those "almost"s aren't a problem in practice. Sure, a list that recursively contains itself looks misleadingly round-trippable, and will evaluate successfully into the wrong thing. But this rarely comes up in practice--and, if it does, because there's a human being inspecting, debugging, or playing with things, rather than a program, it's easy to deal with.
This isn't theoretical--I do this all the time when debugging code, I write my own types to make them easier to debug this way, and it saves me time and hassle. In fact, it's one of the benefits of using Python over some of the other languages I use, where repr or toDebugString or whatever is never useful, instead of being usually useful especially with common types.
The only thing the non-universality of round-tripping means is that you can't use repr with eval as a persistence format. Which is a good thing--you *shouldn't* use it as a persistence format, and that would be true even if it did work. But using it as an inspecting/debugging format is not a problem, and breaking that would be a bad idea.
In fact, breaking it would make repr nearly pointless. Except for a types that define __repr__ but block __str__ with a TypeError (which is rare, and it's debatable whether those types are even valid), when would you ever use repr otherwise?
I don’t think we’re arguing the same point. I’m not saying that __repr__ shouldn’t be round-trippable: if that makes sense for your type then totally fine, go for it. However, I am saying that I don’t see the advantage in having *both* a round-trippable and non-round-trippable repr for the same type. If you’re copying and pasting then the length of the round-trippable representation is a non-issue, and you actively want it to appear in your debug output rather than a shorter version that elides information required to round-trip the data. So in this case, what purpose does the shorter version serve? Cory
On 09.02.2016 12:13, Cory Benfield wrote:
On 9 Feb 2016, at 10:13, Andrew Barnert
wrote: So?
One use of round-trippable reprs is to copy them from output and paste them into source code or an interactive session. When doing so, you almost always know that you're dealing with a builtin or third-party type that's round-trippable--and, when you're surprised, it's almost always obvious, because you get something that looks nothing at all like the source equivalent, and that raises a SyntaxError if you try to evaluate it anyway.
And even those "almost"s aren't a problem in practice. Sure, a list that recursively contains itself looks misleadingly round-trippable, and will evaluate successfully into the wrong thing. But this rarely comes up in practice--and, if it does, because there's a human being inspecting, debugging, or playing with things, rather than a program, it's easy to deal with.
This isn't theoretical--I do this all the time when debugging code, I write my own types to make them easier to debug this way, and it saves me time and hassle. In fact, it's one of the benefits of using Python over some of the other languages I use, where repr or toDebugString or whatever is never useful, instead of being usually useful especially with common types.
The only thing the non-universality of round-tripping means is that you can't use repr with eval as a persistence format. Which is a good thing--you *shouldn't* use it as a persistence format, and that would be true even if it did work. But using it as an inspecting/debugging format is not a problem, and breaking that would be a bad idea.
In fact, breaking it would make repr nearly pointless. Except for a types that define __repr__ but block __str__ with a TypeError (which is rare, and it's debatable whether those types are even valid), when would you ever use repr otherwise? I don’t think we’re arguing the same point. I’m not saying that __repr__ shouldn’t be round-trippable: if that makes sense for your type then totally fine, go for it. However, I am saying that I don’t see the advantage in having *both* a round-trippable and non-round-trippable repr for the same type. If you’re copying and pasting then the length of the round-trippable representation is a non-issue, and you actively want it to appear in your debug output rather than a shorter version that elides information required to round-trip the data. So in this case, what purpose does the shorter version serve?
Same question here. I don't see much use for __short_repr__ being worth the hassle of maintaining possibly two implementations of almost the same thing. All in all, I don't think __repr__ needs optimization in the first place. Best, Sven
On Tue, Feb 9, 2016, at 03:56, Cory Benfield wrote:
I think the reality is that there is no constraint on the representation of arbitrary types to be round-trippable in any way. Again, all custom types have non-round-trippable representations by default, many more eclectic built-in types have non-round-tripppable representations (in addition to NaN, the memoryview object leaps to mind).
One other example is classes and functions though in many cases it's not
clear why this should be the case. Have the default for top-level
functions and classes check whether it's reachable through
[module].[name] and if so return that. The default for methods could use
[class].[name], or [obj].[name] for bound methods.
Instead of the current representation, you could have the default repr
on objects use pickle.
def __repr__(self):
return 'pickle.loads(%r)' % pickle.dumps(self)
Or maybe have a helper function e.g.
def unrepr(description, pickled)
return pickle.dumps(pickled) # ignore description
and default to "unrepr('MyClass object at 0x12345678', b'pickledstuff')"
[Speaking of pickle and since you mentioned memoryview, I noticed a bug:
pickle apparently successfully generates output for a memoryview, but it
fails on attempting to unpickle it - I think that it should throw an
exception immediately on attempting to pickle it.]
Certainly there's the odd edge case where it doesn't make sense for it
to be round-trippable, and there's a tradeoff between human-readability
and round-trippability (see e.g. all the classes that can't be
round-tripped unless the class is imported)... but that's part of where
a short-repr vs long-repr could make sense - short-repr could
Decimal('123.456') or even
On Wednesday, February 10, 2016 12:29 PM, Random832
On Tue, Feb 9, 2016, at 03:56, Cory Benfield wrote:
I think the reality is that there is no constraint on the representation of arbitrary types to be round-trippable in any way. Again, all custom types have non-round-trippable representations by default, many more eclectic built-in types have non-round-tripppable representations (in addition to NaN, the memoryview object leaps to mind).
One other example is classes and functions though in many cases it's not clear why this should be the case. Have the default for top-level functions and classes check whether it's reachable through [module].[name] and if so return that. The default for methods could use [class].[name], or [obj].[name] for bound methods.
Instead of the current representation, you could have the default repr on objects use pickle.
That seems like a bad idea.
First, as a human programmer, the repr "[1, 2, (3, 'spam')]" means something to me--and the same thing to me as to Python; the repr "unrepr('silly.C object at 0x106ec2b38', b'\x80\x03csilly\nC\nq\x00)\x81q\x01}q\x02X\x01\x00\x00\x00xq\x03K\x02sb.')" means less to me than the current "
Mahmoud Hashemi wrote:
I was curious if what kind of interest there would be in adding an overridable method to types to allow getting back a repr, but a predictably short one.
If you just want to know the type and identity of an object, you can use object.__repr__(x). -- Greg
Greg Ewing
Mahmoud Hashemi wrote:
I was curious if what kind of interest there would be in adding an overridable method to types to allow getting back a repr, but a predictably short one.
If you just want to know the type and identity of an object, you can use object.__repr__(x).
Or, to avoid calling dunder methods directly, use ‘type(x)’ and ‘id(x)’. Whether all these suggestions address the stated requirement will depend on how predictable is meant by “predictably short”. -- \ “All persons, living and dead, are purely coincidental.” | `\ —_Timequake_, Kurt Vonnegut | _o__) | Ben Finney
participants (11)
-
Andrew Barnert
-
Ben Finney
-
Chris Barker
-
Cory Benfield
-
Greg Ewing
-
Mahmoud Hashemi
-
Michael Selik
-
Nick Coghlan
-
Random832
-
Steven D'Aprano
-
Sven R. Kunze