Rationale: There are many 2D data with table layout where order is important and it is hard to work with them in Python. The use case: 1. Get data row from a table 2. Change row column by name 3. Save data back For example, termios.tcgetattr() returns list [iflag, oflag, cflag, lflag, ispeed, ospeed, cc]. I need to modify some bits in lflag value and save this list back. In my code it looks like: newattr[3] &= ~termios.ICANON I want to get rid of magic number 3 without defining additional variables just for names. I thought that namedtuple can help here, but it is read-only and seems not serializeable. What I want is: >>> data = [1, 'john', 23434] >>> named = Record(data, ['id', 'name', 'number']) >>> named.id = 3 >>> named.name 'john' >>> named [3, 'john', 23434] >>> named.naem AttributeError(...) >>> named.dict() {'id': 3, 'name': john, 'number': 23434} >>> named.json() ... I hacked OrderedDict to accept list as param and allow attribute access. It doesn't behave as named list - you still need to call .values() to get list back, and indexed access doesn't work as well. http://bugs.python.org/file31026/DictRecord.py -- anatoly t.
On Wed, Jul 24, 2013 at 10:50 AM, anatoly techtonik
newattr[3] &= ~termios.ICANON
I want to get rid of magic number 3 without defining additional variables just for names. I thought that namedtuple can help here, but it is read-only and seems not serializeable.
This indeed would be useful in the csv module as DictReader/DictWriter are cute but the [""] are a bit extraneous. And it'd be a useful impromptu, mutable replacement for namedtuple. The only minor problem I foresee is when you have illegal python words as headers (e.g. named.class or named.break) but then you can fall back to [""]. But now you have more than one way to do it. Yuval
On 24 Jul, 2013, at 9:50, anatoly techtonik
Rationale: There are many 2D data with table layout where order is important and it is hard to work with them in Python.
The use case: 1. Get data row from a table 2. Change row column by name 3. Save data back
For example, termios.tcgetattr() returns list [iflag, oflag, cflag, lflag, ispeed, ospeed, cc]. I need to modify some bits in lflag value and save this list back. In my code it looks like:
newattr[3] &= ~termios.ICANON
I want to get rid of magic number 3 without defining additional variables just for names. I thought that namedtuple can help here, but it is read-only and seems not serializeable.
Namedtuple has an _update method, which would allow you to write the code below if tcgetattr returns a named tuple: newattr = newattr._replace(cflag=newattr.cflag & ~termios.ICANON) However, tcgetattr does not return a namedtuple :-) What do you mean by "namedtuple [...] seems not serializable"? Both pickle and json work fine with named tuples. Ronald
On Wed, Jul 24, 2013 at 12:25 PM, Ronald Oussoren
Namedtuple has an _update method, which would allow you to write the code below if tcgetattr returns a named tuple:
newattr = newattr._replace(cflag=newattr.cflag & ~termios.ICANON)
However, tcgetattr does not return a namedtuple :-)
Converting is possible, even if it looks ugly to call private method outside of class, but it is also unclear if tcsetattr() can accept nametuples.
What do you mean by "namedtuple [...] seems not serializable"? Both pickle and json work fine with named tuples.
While searching archive for similar topic I've found this one, which says that namedtuple pickling is "sometimes possible": https://groups.google.com/forum/#!topic/python-ideas/Pw0hNdiTu8A -- anatoly t.
On 24/07/13 17:50, anatoly techtonik wrote:
Rationale: There are many 2D data with table layout where order is important and it is hard to work with them in Python. [...]
It sounds to me that what you want is a mutable namedtuple. At least, that's what I often want. namedtuple was a fantastically useful addition to the standard library, I think a mutable record type would be too. Here's a quick and dirty (emphasis on the dirty) proof of concept of the sort of thing I'd like: def record(name, fields): def __init__(self, *args): for slot, arg in zip(self.__slots__, args): setattr(self, slot, arg) return type(name, (), {'__slots__': fields.split(), '__init__': __init__} ) I don't put this forward as a production-ready solution, it is missing a nice repr, and doesn't support indexing or sequence unpacking, and I'm not really sure if it should use slots, but it gives the idea of the sort of thing that can be done. [...]
I hacked OrderedDict to accept list as param and allow attribute access. It doesn't behave as named list - you still need to call .values() to get list back, and indexed access doesn't work as well.
I don't think that OrderedDict is a good base class for this. Dicts have lots of methods that are completely inappropriate for a record/struct-like object. -- Steven
On 07/24/2013 08:54 AM, Steven D'Aprano wrote:
On 24/07/13 17:50, anatoly techtonik wrote:
Rationale: There are many 2D data with table layout where order is important and it is hard to work with them in Python. [...]
It sounds to me that what you want is a mutable namedtuple. At least, that's what I often want. namedtuple was a fantastically useful addition to the standard library, I think a mutable record type would be too.
There are a number of implementations of this on PyPi, and probably elsewhere. Here's one I wrote: https://pypi.python.org/pypi/recordtype/ It could use some polishing. I don't particularly like that it conflates mutability along with default parameters, but I couldn't think of an easy way to separate them, and for my particular use I needed both. Eric.
While we're on the subject of possible alternatives, I think macropy's case
classes https://github.com/lihaoyi/macropy#case-classes are a pretty nice
balance between namedtuple-ish concerns of conciseness, auto-{__str__,
__eq__, __init__, etc.} as well as class-ish concerns of being able to give
them methods and initializers, with a pretty nice (macro-powered) syntax.
It's a syntax that looks equally pretty whether you're writing a simple
Point(x, y) struct or a large-ish class with many members, initialization
logic and methods, which I think is pretty nice.
On Wed, Jul 24, 2013 at 9:28 PM, Eric V. Smith
On 07/24/2013 08:54 AM, Steven D'Aprano wrote:
On 24/07/13 17:50, anatoly techtonik wrote:
Rationale: There are many 2D data with table layout where order is important and it is hard to work with them in Python. [...]
It sounds to me that what you want is a mutable namedtuple. At least, that's what I often want. namedtuple was a fantastically useful addition to the standard library, I think a mutable record type would be too.
There are a number of implementations of this on PyPi, and probably elsewhere. Here's one I wrote:
https://pypi.python.org/pypi/recordtype/
It could use some polishing. I don't particularly like that it conflates mutability along with default parameters, but I couldn't think of an easy way to separate them, and for my particular use I needed both.
Eric.
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Wed, Jul 24, 2013 at 4:28 PM, Eric V. Smith
On 07/24/2013 08:54 AM, Steven D'Aprano wrote:
On 24/07/13 17:50, anatoly techtonik wrote:
Rationale: There are many 2D data with table layout where order is important and it is hard to work with them in Python. [...]
It sounds to me that what you want is a mutable namedtuple. At least, that's what I often want. namedtuple was a fantastically useful addition to the standard library, I think a mutable record type would be too.
Yes, "mutable namedtuple" is the name.
There are a number of implementations of this on PyPi, and probably elsewhere. Here's one I wrote:
https://pypi.python.org/pypi/recordtype/
It could use some polishing. I don't particularly like that it conflates mutability along with default parameters, but I couldn't think of an easy way to separate them, and for my particular use I needed both.
That was the first idea for me too, but I'd like to see this defined differently. I remember my confusion when I first met "namedtuple" factory call in the code, so I'd prefer an easy static approach for names definition (such as class attribute). Default are not needed for such "mutable namedtuple", because it should always be initialized with list of the fixed length. What does the recordtype module do is helping to build a record given only partial set of data fields. This use case is often seen in DB oriented applications (Django models [1], etc.) and if you want a clear definition of your defaults, IMHO that's a way to go. Other static definitions I can think about look ugly and not extensible (i.e. with additional field properties besides defaults): class TermiosState(DictRecord): NAMES = ['iflag', 'oflag', 'cflag', 'lflag', 'ispeed', 'ospeed', 'cc'] DEFAULTS = [0, 0, 0, 0, 0, 0, 0] or class TermiosState(DictRecord): NAMES = ['iflag', 'oflag', 'cflag', 'lflag', 'ispeed', 'ospeed', 'cc'] DEFAULTS = dict(lflag=0) or class TermiosState(DictRecord): NAMES = dict( iflag=0, oflag=0, ... ) I like the definition of macropy "case" classes [2] mentioned by Haoyi Li. The name is the most unfortunate among all the names I can remember - "case" means so many things in English and in programming in particular, that people new to the concept may hack their brain before they find an appropriate tutorial. These "case classes" surely look like a hack, as they redefine the role of class definition parameters, but they hold more practical value than a multiple inheritance to me: @case class Point(x, y): pass The TermiosState can then be rewritten as: @record class TermiosState(iflag, oflag, cflag, lflag, ispeed, ospeed, cc): iflag = 0 # default Of course this is not as extensible as Django model definition. I am not sure if such syntax is portable across Python implementations, but it is the most concise. Of course there should be the proper check for name typos in section with defaults. 1. https://docs.djangoproject.com/en/dev/topics/db/models/ 2. https://github.com/lihaoyi/macropy#case-classes -- anatoly t.
On Wed, Jul 24, 2013 at 3:54 PM, Steven D'Aprano
Here's a quick and dirty (emphasis on the dirty) proof of concept of the sort of thing I'd like:
def record(name, fields): def __init__(self, *args): for slot, arg in zip(self.__slots__, args): setattr(self, slot, arg) return type(name, (), {'__slots__': fields.split(), '__init__': __init__} )
I don't put this forward as a production-ready solution, it is missing a nice repr, and doesn't support indexing or sequence unpacking, and I'm not really sure if it should use slots, but it gives the idea of the sort of thing that can be done.
Indexing is vital for lists, the idea is to have a list replacement, just with convenient names to write more readable code. Slots looks like a good solution to throw errors about typos in attribute names as early as possible.
[...]
I hacked OrderedDict to accept list as param and allow attribute access. It doesn't behave as named list - you still need to call .values() to get list back, and indexed access doesn't work as well.
I don't think that OrderedDict is a good base class for this. Dicts have lots of methods that are completely inappropriate for a record/struct-like object.
I agree, but it was the only way I can think of to get clean definition for list fields, such as: class TermiosState(DictRecord): NAMES = ['iflag', 'oflag', 'cflag', 'lflag', 'ispeed', 'ospeed', 'cc'] This definition can be easily parsed by static analyzer tools like pyflakes. The standard nametuple definition is not so clear to them: Point = namedtuple('Point', ['x', 'y'], verbose=True) -- anatoly t.
participants (6)
-
anatoly techtonik -
Eric V. Smith -
Haoyi Li -
Ronald Oussoren -
Steven D'Aprano -
Yuval Greenfield