Mailman 3 January 2013 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley 17 Dec '20

17 Dec '20

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann 22 Feb '20

22 Feb '20

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int array of the same size (8 bytes per number, so it occupies about 48 GB memory). At the moment I am extending an array in chunks of several million items at a time at a time, which is slow and not elegant. The function below also initializes each item in the array to a given value (0 by default). Is there a reason why there the array.array constructor does not allow to simply specify the number of items that should be allocated? (I do not really care about the contents.) Would this be a worthwhile addition to / modification of the array module? My suggestions is to modify array generation in such a way that you could pass an iterator (as now) as second argument, but if you pass a single integer value, it should be treated as the number of items to allocate. Here is my current workaround (which is slow): def filled_array(typecode, n, value=0, bsize=(1<<22)): """returns a new array with given typecode (eg, "l" for long int, as in the array module) with n entries, initialized to the given value (default 0) """ a = array.array(typecode, [value]*bsize) x = array.array(typecode) r = n while r >= bsize: x.extend(a) r -= bsize x.extend([value]*r) return x

14 20

while conditional in list comprehension ??
by Wolfgang Maier 21 Feb '14

21 Feb '14

Dear all, I guess this is so obvious that someone must have suggested it before: in list comprehensions you can currently exclude items based on the if conditional, e.g.: [n for n in range(1,1000) if n % 4 == 0] Why not extend this filtering by allowing a while statement in addition to if, as in: [n for n in range(1,1000) while n < 400] Trivial effect, I agree, in this example since you could achieve the same by using range(1,400), but I hope you get the point. This intuitively understandable extension would provide a big speed-up for sorted lists where processing all the input is unnecessary. Consider this: some_names=["Adam", "Andrew", "Arthur", "Bob", "Caroline","Lancelot"] # a sorted list of names [n for n in some_names if n.startswith("A")] # certainly gives a list of all names starting with A, but . [n for n in some_names while n.startswith("A")] # would have saved two comparisons Best, Wolfgang

19 70

format specifier for "not bytes"
by Daniel Holth 06 Jul '13

06 Jul '13

While I was implementing JSON-JWS (JSON web signatures), a format which in Python 3 has to go from bytes > unicode > bytes > unicode several times in its construction, I notice I wrote a lot of bugs: "sha256=b'abcdef1234'" When I meant to say: "sha256=abcdef1234" Everything worked perfectly on Python 3 because the verifying code also generated the sha256=b'abcdef1234' as a comparison. I would have never noticed at all unless I had tried to verify the Python 3 output with Python 2. I know I'm a bad person for not having unit tests capable enough to catch this bug, a bug I wrote repeatedly in each layer of the bytes > unicode > bytes > unicode dance, and that there is no excuse for being confused at any time about the type of a variable, but I'm not willing to reform. Instead, I would like a new string formatting operator tentatively called 'notbytes': "sha256=%notbytes" % (b'abcdef1234'). It gives the same error as 'sha256='+b'abc1234' would: TypeError: Can't convert 'bytes' object to str implictly

8 17

hexdump
by anatoly techtonik 29 Apr '13

29 Apr '13

Just an idea of usability fix for Python 3. hexdump module (function or bytes method is better) as simple, easy and intuitive way for dumping binary data when writing programs in Python. hexdump(bytes) - produce human readable dump of binary data, byte-by-byte representation, separated by space, 16-byte rows Rationale: 1. Debug. Generic binary data can't be output to console. A separate helper is needed to print, log or store its value in human readable format in database. This takes time. 2. Usability. binascii is ugly: name is not intuitive any more, there are a lot of functions, and it is not clear how it relates to unicode. 3. Serialization. It is convenient to have format that can be displayed in a text editor. Simple tools encourage people to use them. Practical example: >>> print(b) � � � �� >>> b '\xe6\xb0\x08\x04\xe7\x9e\x08\x04\xe7\xbc\x08\x04\xe7\xd5\x08\x04\xe7\xe4\x08\x04\xe6\xb0\x08\x04\xe7\xf0\x08\x04\xe7\xff\x08\x04\xe8\x0b\x08\x04\xe8\x1a\x08\x04\xe6\xb0\x08\x04\xe6\xb0\x08\x04' >>> print(binascii.hexlify(data)) e6b00804e79e0804e7bc0804e7d50804e7e40804e6b00804e7f00804e7ff0804e80b0804e81a0804e6b00804e6b00804 >>> >>> data = hexdump(b) >>> print(data) E6 B0 08 04 E7 9E 08 04 E7 BC 08 04 E7 D5 08 04 E7 E4 08 04 E6 B0 08 04 E7 F0 08 04 E7 FF 08 04 E8 0B 08 04 E8 1A 08 04 E6 B0 08 04 E6 B0 08 04 >>> >>> # achieving the same output with binascii is overcomplicated >>> data_lines = [binascii.hexlify(b)[i:min(i+32, len(binascii.hexlify(b)))] for i in xrange(0, len(binascii.hexlify(b)), 32)] >>> data_lines = [' '.join(l[i:min(i+2, len(l))] for i in xrange(0, len(l), 2)).upper() for l in data_lines] >>> print('\n'.join(data_lines)) E6 B0 08 04 E7 9E 08 04 E7 BC 08 04 E7 D5 08 04 E7 E4 08 04 E6 B0 08 04 E7 F0 08 04 E7 FF 08 04 E8 0B 08 04 E8 1A 08 04 E6 B0 08 04 E6 B0 08 04 On the other side, getting rather useless binascii output from hexdump() is quite trivial: >>> data.replace(' ','').replace('\n','').lower() 'e6b00804e79e0804e7bc0804e7d50804e7e40804e6b00804e7f00804e7ff0804e80b0804e81a0804e6b00804e6b00804' But more practical, for example, would be counting offset from hexdump: >>> print( ''.join( '%05x: %s\n' % (i*16,l) for i,l in enumerate(hexdump(b).split('\n')))) Etc. Conclusion: By providing better building blocks on basic level Python will become a better tool for more useful tasks. References: [1] http://stackoverflow.com/questions/2340319/python-3-1-1-string-to-hex [2] http://en.wikipedia.org/wiki/Hex_dump -- anatoly t.

8 10

constant/enum type in stdlib
by Joao S. O. Bueno 13 Feb '13

13 Feb '13

This idea is not new - but it is stalled - Last I remember it came around in Python-devel in 2010, in this thread: http://mail.python.org/pipermail/python-dev/2010-November/thread.html#105967 There is an even older PEP (PEP 354) that was rejected just for not being enough interest at the time - And it was not dismissed at all - to the contrary the last e-mail in the thread is a message from the BDLF for it to **be** ! The discussion happened in a bad moment as Python was mostly freature froozen for 3.2 - and it did not show up again for Python 3.3; The reasoning for wanting enums/ constants has been debated already - but one of the main reasons that emerge from that thread are the ability to have named constants (just like we have "True" and "False". why do I think this is needed in the stdlib, and having itin a 3rd party module is not enough? because they are an interesting thing to have, not only on the stdlib, but on several widely used Python projects that don't have other dependencies. Having a feature like this into the stdlib allow these projects to make use of it, without needing other dependencies, and moreover, users which will benefit the most out of such constants will have a wll known "constant" type which won't come as a surprise in each package he is using interactively or debugging. Most of the discussion on the 2010 thread was summed up in a message by Michael Foord in this link http://mail.python.org/pipermail/python-dev/2010-November/106063.html with some follow up here: http://mail.python.org/pipermail/python-dev/2010-November/106065.html js -><- ---------- --

23 80

libuv based eventloop for tulip experiment
by Saúl Ibarra Corretgé 08 Feb '13

08 Feb '13

Hi all! I haven't been able to keep up with all the tulip development on the mailing list (hopefully I will now!) so please excuse me if something I mention has already been discussed. For those who may not know it, libuv is the platform layer library for nodejs, which implements a uniform interface on top of epoll, kqueue, event ports and iocp. I wrote Python bindings [1] for it a while ago, and I was very excited to see Tulip, so I thought I'd give this a try. Here [2] is the source code, along with some notes I took during the implementation. I know that the idea is not to re-implement the PEP itself but for people to create different EventLoop implementations. On rose I bundled tulip just to make a single package I could play with easily, once tulip makes it to the stdlib only the EventLop will remain. Here are some thoughts (in no particular order): - add_connector / remove_connector seem to be related to Windows, but being exposed like that feels a bit like leaking an implementation detail. I guess there was no way around it. - libuv implements a type of handle (Poll) which provides level-triggered file descriptor polling which also works on Windows, while being highly performant. It uses something called AFD Polling apparently, which is only available on Windows >= Vista, and a select thread on XP. I'm no Windows expert, but thanks to this the API is consistent across all platforms, which is nice. mAybe it's worth investigating? [3] - The transport abstraction seems quite tight to socket objects. pyuv provides a TCP and UDP handles, which provide a completion-style API and use a better approach than Poll handles. They should give better performance since EINTR in handled internally and there are less roundtrips between Python-land and C-land. Was it ever considered to provide some sort of abstraction so that transports can be used on top of something other than regular sockets? For example I see no way to get the remote party from the transport, without checking the underlying socket. Thanks for reading this far and keep up the good work. Regards, [1]: https://github.com/saghul/pyuv [2]: https://github.com/saghul/rose [3]: https://github.com/joyent/libuv/blob/master/src/win/poll.c -- Saúl Ibarra Corretgé http://saghul.net/blog | http://about.me/saghul

4 15

Extend module objects to support properties
by Larry Hastings 05 Feb '13

05 Feb '13

Properties are a wonderful facility. But they only work on conventional objects. Specifically, they *don't* work on module objects. It would be nice to extend module objects so properties worked there too. For example, Victor Stinner's currently proposed PEP 433 adds two new methods to the sys module: sys.getdefaultcloexc() and sys.setdefaultcloexc(). What are we, Java? Surely this would be much nicer as a property, sys.defaultcloexc. //arry/

10 20

Re: [Python-ideas] PEP 3156: getting the socket or peer name from the transport
by Antoine Pitrou 02 Feb '13

02 Feb '13

Le dimanche 27 janvier 2013 à 14:12 +0200, Yuval Greenfield a écrit : > On Sun, Jan 27, 2013 at 1:21 PM, Antoine Pitrou <solipsis(a)pitrou.net> > wrote: > > Most protocols should be written independent of transport. > But it seems to > > me that a user might write an entire app as a "protocol". > > > Well, such an assumption can fall flat. For example, > certificate > checking in HTTPS expects that the transport is some version > of TLS or > SSL: http://tools.ietf.org/html/rfc2818.html#section-3.1 > > > I'm not sure I understood your reply. You'd be for an api that exposes > the underlying transport? I meant to say that "an entire app" entails > control over the subtleties of the underlying transport. What I meant is that the HTTP protocol needs to know that it is running over a secure transport, and it needs to fetch the server certificate from that transport (or, alternatively, it needs to have one of its callbacks called by the transport when the certificate is known). That's not entirely transport-agnostic. Regards Antoine.

4 9

Definition Symmetry
by Jason Keene 02 Feb '13

02 Feb '13

Why do function definitions require parens? >>> class MyClass: ... pass ... >>> def my_func: File "<stdin>", line 1 def my_func: ^ SyntaxError: invalid syntax This seems to me to break a symmetry with class definitions. I assume this is just a hold off from C, perhaps there is a non-historical reason tho. I believe in the past we've forced parens in list comprehensions to create a symmetry between comprehensions/generator expressions. Why not for this?

12 16

Python-ideas January 2013