I just tried running regrtest with "-uall,-largefile" (after a "cvs up", "./config.status --recheck", and "make") on my Mac OS X system. It chugged for awhile, then spit this out several times: Exception in thread reader 4: Traceback (most recent call last): File "/Users/skip/src/python/head/dist/src/Lib/threading.py", line 411, in __bootstrap self.run() File "/Users/skip/src/python/head/dist/src/Lib/threading.py", line 399, in run self.__target(*self.__args, **self.__kwargs) File "/Users/skip/src/python/head/dist/src/Lib/bsddb/test/test_thread.py", line 270, in reade rThread rec = c.first() DBLockDeadlockError: (-30995, 'DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock') once for each thread, then this: /Users/skip/src/python/head/dist/src/Lib/bsddb/dbutils.py:67: RuntimeWarning: DB_INCOMPLETE: Ca che flush was unable to complete return function(*_args, **_kwargs) After chugging awhile longer, it segfaulted. What (if anything) can I do to provide useful inputs to someone who can possibly fix the problem? Skip
There are many places in the standard library where some code either iterates over a literal list or checks for membership in a literal list. I'm curious if it would be considered productive and useful to go through and change those places to iterate over/check for membership in literal tuples instead fo lists. The tuple, I think, more closely reflects the read-only literal nature of the code and is slightly faster to boot. (Not that the speed really matters, I'm sure there aren't any such tests in performance-sensitive locations). Would such an endeavor be useful? Would a patch to that effect be accepted? Jeremy
-1. I bet you can't prove the speed-up. Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists. Tuples require extra care in case the number of elements shrinks to 1. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tuesday 11 March 2003 09:11 pm, Guido van Rossum wrote:
I bet you can't prove the speed-up.
Here's the script I used to test it: import timeit report = \ """ Size: %s Tuple Time: %s List Time: %s List Time - Tuple Time: %s """ if __name__ == '__main__': import sys if len(sys.argv) > 1: upperLimit = sys.argv[1] else: upperLimit = 10 for i in xrange(upperLimit): lst = range(i) tpl = tuple(lst) tupleTimer = timeit.Timer('%s in %r' % (upperLimit, tpl)) listTimer = timeit.Timer('%s in %r' % (upperLimit, lst)) minTupleTime = min(tupleTimer.repeat()) minListTime = min(listTimer.repeat()) difference = minListTime - minTupleTime print report % (i, minTupleTime, minListTime, difference) There seems to be a constant 1.3 usec or so difference between creating a tuple and creating a list. As I mentioned earlier, I seriously doubt it would have any significant impact on the overall execution speed of any non-trivial Python program, but it exists nonetheless. Maybe in the realm of 'low hanging fruit' it's the fruit that's fallen to the ground and begun to rot :)
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
I understand this in a strictly typed language, but in Python, since lists can be just as heterogeneous as tuples, it's always seemed to me that the greatest difference between lists and tuples is the mutability and extensibility of lists. Jeremy
[Good use of 'timeit' module skipped]
Of course creating a tuple is faster than creating a list. I meant that you wouldn't be able to show a speed difference in any of the places where you would consider adding it (i.e. in context).
Sorry, you're wrong. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote: ...
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
Oh! Did you point that out anywhere, before, and I missed it? Are you thinking of lists as to be really somehow being homogeneous data, in a sense to be replacible by some array optimization, sometimes, while tuples aren't? I never realized this, and I'm a bit stunned. (but by no means negative about it, just surprized) ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Yes. I've been saying this for years whenever people would listen (which is not often :-( )
Python is a dynamic language, and you can do whatever you want with the data structures it gives you. But when thinking about extending the language with optional type declarations or automatic type inference, I always think of the type of a list as "list of T" while I think of a tuple's type as "tuple of length N with items of types T1, T2, T3, ..., TN". So [1, 2] and [1, 2, 3] are both "list of int" (and "list of Number" and "list of Object", of course) while ("hello", 42) is a "2-tuple with items str and int" and (42, "hello", 3.14) is a "3-tuple with items int, str, float".
I never realized this, and I'm a bit stunned. (but by no means negative about it, just surprized)
You learn something new every day. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido> Python is a dynamic language, and you can do whatever you want Guido> with the data structures it gives you. But when thinking about Guido> extending the language with optional type declarations or Guido> automatic type inference, I always think of the type of a list Guido> as "list of T" while I think of a tuple's type as "tuple of Guido> length N with items of types T1, T2, T3, ..., TN". So [1, 2] Guido> and [1, 2, 3] are both "list of int" (and "list of Number" and Guido> "list of Object", of course) while ("hello", 42) is a "2-tuple Guido> with items str and int" and (42, "hello", 3.14) is a "3-tuple Guido> with items int, str, float". It might interest you to know that Standard ML, which is statically but polymorphically typed, draws exactly that distinction. It has both tuple and list types. The type of a tuple includes the type of each of its elements, whereas all of the elements of a list must be the same type. Moreover, although the type of a list includes the type of its elements, it does not include how many elements there are. So in ML, the type of [1, 2, 3] is "int list", and the type of ("hello", 42) is "string * int". -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark
"GvR" == Guido van Rossum <guido@python.org> writes:
GvR> I always think of the type of a list as "list of T" while I GvR> think of a tuple's type as "tuple of length N with items of GvR> types T1, T2, T3, ..., TN". So [1, 2] and [1, 2, 3] are both GvR> "list of int" (and "list of Number" and "list of Object", of GvR> course) while ("hello", 42) is a "2-tuple with items str and GvR> int" and (42, "hello", 3.14) is a "3-tuple with items int, GvR> str, float". Of course (1, 2, 3) fits under that description, where, just by chance <wink> T1 == T2 == T3. But one of the ways I think about it is the tuple's relationship to argument and return passing. It's the tuple that's used when multiple values are returned from a function and they are almost always heterogeneous. And while lists can be used for unpacking sequences, I tend to think of tuples when I want record types, e.g. rec = magic(blah) length, prefix, interface = rec -Barry
Guido van Rossum wrote:
Sorry.
Oh yes, after re-thinking this, my question was dumb. From my own usage of tuples and lists, I know that I almost always use lists as collections of objects of the same type, while tuples are often used to group different things together. Basically, I knew this all, and I'm asking myself why I asked. Probably since I'm looking at lists and tuples too much technically, these days. cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Guido:
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
Weird things are happening in my brain this morning. After reading this thread, I was replying to something unrelated and had the occasion to use the phrase "It's on my list"... and I briefly wondered whether I should use the word "tuple" instead! Somehow "It's on my tuple" doesn't have quite the same ring to it. So, yes, tuples ARE different from lists... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Guido van Rossum wrote:
He probably can :-) That's why I have so many tools in mxTools which return tuples instead of lists, e.g. trange() and indices(). Both the tuple creation and the iteration are faster than list creation and access (tuples don't use indirection which saves you a second malloc() and dereference). As always: it's the sum of small tweaks like these that makes the difference in the overall performance of an application. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Mar 12 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
Python UK 2003, Oxford: 20 days left EuroPython 2003, Charleroi, Belgium: 104 days left
On Tue, Mar 11, 2003, Guido van Rossum wrote:
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
It's been on my To-Do list to update PEP 8 since last June; if someone else wants to do it, be my guest. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Register for PyCon now! http://www.python.org/pycon/reg.html
[Skip Montanaro]
I believe those are timing-related and harmless. It would be better if the test suite suppressed such msgs, if so.
That's not good, though.
After chugging awhile longer, it segfaulted.
Nor that.
What (if anything) can I do to provide useful inputs to someone who can possibly fix the problem?
Sorry, no idea. Is the pybsddb SF project still open for business?
There are many places in the standard library where some code either iterates over a literal list or checks for membership in a literal list. I'm curious if it would be considered productive and useful to go through and change those places to iterate over/check for membership in literal tuples instead fo lists. The tuple, I think, more closely reflects the read-only literal nature of the code and is slightly faster to boot. (Not that the speed really matters, I'm sure there aren't any such tests in performance-sensitive locations). Would such an endeavor be useful? Would a patch to that effect be accepted? Jeremy
-1. I bet you can't prove the speed-up. Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists. Tuples require extra care in case the number of elements shrinks to 1. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tuesday 11 March 2003 09:11 pm, Guido van Rossum wrote:
I bet you can't prove the speed-up.
Here's the script I used to test it: import timeit report = \ """ Size: %s Tuple Time: %s List Time: %s List Time - Tuple Time: %s """ if __name__ == '__main__': import sys if len(sys.argv) > 1: upperLimit = sys.argv[1] else: upperLimit = 10 for i in xrange(upperLimit): lst = range(i) tpl = tuple(lst) tupleTimer = timeit.Timer('%s in %r' % (upperLimit, tpl)) listTimer = timeit.Timer('%s in %r' % (upperLimit, lst)) minTupleTime = min(tupleTimer.repeat()) minListTime = min(listTimer.repeat()) difference = minListTime - minTupleTime print report % (i, minTupleTime, minListTime, difference) There seems to be a constant 1.3 usec or so difference between creating a tuple and creating a list. As I mentioned earlier, I seriously doubt it would have any significant impact on the overall execution speed of any non-trivial Python program, but it exists nonetheless. Maybe in the realm of 'low hanging fruit' it's the fruit that's fallen to the ground and begun to rot :)
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
I understand this in a strictly typed language, but in Python, since lists can be just as heterogeneous as tuples, it's always seemed to me that the greatest difference between lists and tuples is the mutability and extensibility of lists. Jeremy
[Good use of 'timeit' module skipped]
Of course creating a tuple is faster than creating a list. I meant that you wouldn't be able to show a speed difference in any of the places where you would consider adding it (i.e. in context).
Sorry, you're wrong. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote: ...
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
Oh! Did you point that out anywhere, before, and I missed it? Are you thinking of lists as to be really somehow being homogeneous data, in a sense to be replacible by some array optimization, sometimes, while tuples aren't? I never realized this, and I'm a bit stunned. (but by no means negative about it, just surprized) ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Yes. I've been saying this for years whenever people would listen (which is not often :-( )
Python is a dynamic language, and you can do whatever you want with the data structures it gives you. But when thinking about extending the language with optional type declarations or automatic type inference, I always think of the type of a list as "list of T" while I think of a tuple's type as "tuple of length N with items of types T1, T2, T3, ..., TN". So [1, 2] and [1, 2, 3] are both "list of int" (and "list of Number" and "list of Object", of course) while ("hello", 42) is a "2-tuple with items str and int" and (42, "hello", 3.14) is a "3-tuple with items int, str, float".
I never realized this, and I'm a bit stunned. (but by no means negative about it, just surprized)
You learn something new every day. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido> Python is a dynamic language, and you can do whatever you want Guido> with the data structures it gives you. But when thinking about Guido> extending the language with optional type declarations or Guido> automatic type inference, I always think of the type of a list Guido> as "list of T" while I think of a tuple's type as "tuple of Guido> length N with items of types T1, T2, T3, ..., TN". So [1, 2] Guido> and [1, 2, 3] are both "list of int" (and "list of Number" and Guido> "list of Object", of course) while ("hello", 42) is a "2-tuple Guido> with items str and int" and (42, "hello", 3.14) is a "3-tuple Guido> with items int, str, float". It might interest you to know that Standard ML, which is statically but polymorphically typed, draws exactly that distinction. It has both tuple and list types. The type of a tuple includes the type of each of its elements, whereas all of the elements of a list must be the same type. Moreover, although the type of a list includes the type of its elements, it does not include how many elements there are. So in ML, the type of [1, 2, 3] is "int list", and the type of ("hello", 42) is "string * int". -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark
"GvR" == Guido van Rossum <guido@python.org> writes:
GvR> I always think of the type of a list as "list of T" while I GvR> think of a tuple's type as "tuple of length N with items of GvR> types T1, T2, T3, ..., TN". So [1, 2] and [1, 2, 3] are both GvR> "list of int" (and "list of Number" and "list of Object", of GvR> course) while ("hello", 42) is a "2-tuple with items str and GvR> int" and (42, "hello", 3.14) is a "3-tuple with items int, GvR> str, float". Of course (1, 2, 3) fits under that description, where, just by chance <wink> T1 == T2 == T3. But one of the ways I think about it is the tuple's relationship to argument and return passing. It's the tuple that's used when multiple values are returned from a function and they are almost always heterogeneous. And while lists can be used for unpacking sequences, I tend to think of tuples when I want record types, e.g. rec = magic(blah) length, prefix, interface = rec -Barry
Guido van Rossum wrote:
Sorry.
Oh yes, after re-thinking this, my question was dumb. From my own usage of tuples and lists, I know that I almost always use lists as collections of objects of the same type, while tuples are often used to group different things together. Basically, I knew this all, and I'm asking myself why I asked. Probably since I'm looking at lists and tuples too much technically, these days. cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Guido:
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
Weird things are happening in my brain this morning. After reading this thread, I was replying to something unrelated and had the occasion to use the phrase "It's on my list"... and I briefly wondered whether I should use the word "tuple" instead! Somehow "It's on my tuple" doesn't have quite the same ring to it. So, yes, tuples ARE different from lists... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Guido van Rossum wrote:
He probably can :-) That's why I have so many tools in mxTools which return tuples instead of lists, e.g. trange() and indices(). Both the tuple creation and the iteration are faster than list creation and access (tuples don't use indirection which saves you a second malloc() and dereference). As always: it's the sum of small tweaks like these that makes the difference in the overall performance of an application. -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Mar 12 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
Python UK 2003, Oxford: 20 days left EuroPython 2003, Charleroi, Belgium: 104 days left
On Tue, Mar 11, 2003, Guido van Rossum wrote:
Tuples are for heterogeneous data, list are for homogeneous data. Tuples are *not* read-only lists.
It's been on my To-Do list to update PEP 8 since last June; if someone else wants to do it, be my guest. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Register for PyCon now! http://www.python.org/pycon/reg.html
[Skip Montanaro]
I believe those are timing-related and harmless. It would be better if the test suite suppressed such msgs, if so.
That's not good, though.
After chugging awhile longer, it segfaulted.
Nor that.
What (if anything) can I do to provide useful inputs to someone who can possibly fix the problem?
Sorry, no idea. Is the pybsddb SF project still open for business?
participants (11)
-
Aahz -
Andrew Koenig -
barry＠python.org -
Christian Tismer -
Greg Ewing -
Guido van Rossum -
Jeremy Fincher -
M.-A. Lemburg -
Neil Schemenauer -
Skip Montanaro -
Tim Peters