Mailman 3 October 2018 - Python-ideas

Set starting point for itertools.product()
by Ronie Martinez Oct. 26, 2018

Oct. 26, 2018

Hi, My idea is to set the starting point for itertools.product() <https://docs.python.org/3.6/library/itertools.html#itertools.product> since it becomes very slow if the point of interest is in the middle. For example when working with datetime tuples with seconds resolution (worst case, milli/microseconds), you need to skip a lot of items. There are several methods I have tried as an alternative but itertools.product() is the fastest: - datetime and timedelta - naive looping This is where I applied it but the issue at seconds resolution is still a bottleneck: https://github.com/Code-ReaQtor/DocCron/blob/1.0.0/doccron/job.py#L171 Setting the starting point might come in handy if we want to skip several data. What do you think? Best Regards, Ronie

5 10

Contracts in python -- a report & next steps
by Marko Ristin-Kaufmann Oct. 25, 2018

Oct. 25, 2018

Hi, I would like to give you a short report on the development of icontract library following the discussion about the introduction of contracts into Python (e.g., see [1, 2, 3, 4]). *Features* The functionality of icontract library is basically on par with Eiffel, Cobra and other languages supporting contracts except for loop invariants and less readable syntax. The features include: * "require", "ensure" and "invariant" decorators to define the contracts * "snapshot" decorator to capture the values prior to the function invocation to allow postconditions to verify the state transitions * inheritance of the contracts (with strengthening/weakening) * tracing of the argument values on contract violation * informative violation messages automatically parsed from the code of the condition function * The user can specify custom errors to be raised on a specific contract violation * individual toggling of the contracts * linter to check the contract arguments * sphinx extension to render the contracts automatically in the documentation (including a sophisticated matching of logical implications) We covered all the use cases we could find in our code base (at Parquery AG) such as: * functions * instance methods * class and static methods * property getters, setters and deleters * slot wrappers *Roadblocks* During the development, the following roadblocks were encountered: * We wanted to include the contracts in the output of help(). Unfortunately, help() renders the __doc__ of the class and not of the instance. For functions, this is the class "function" which you can not inherit from. See [5] for more details. * We need to inspect the source code of the condition and error lambdas to generate the violation message and infer the error type in the documentation, respectively. inspect.getsource(.) is broken on lambdas defined in decorators in Python 3.5.2+ (see [6]). We circumvented this bug by using inspect.findsource(.), inspect.getsourcefile(.) and examining the local source code of the lambda by searching for other decorators above and other decorators and a function or class definition below. The decorator code is parsed and then we match the condition and error arguments in the AST of the decorator. This is brittle as it prevents us from having partial definitions of contract functions or from sharing the contracts among functions. Here is a short code snippet to demonstrate where the current implementation fails: import icontract require_x_positive = icontract.require( lambda x: x > 0, error=lambda: ValueError("x must be positive")) @require_x_positive def some_func(x: int) -> None: pass However, we haven't faced a situation in the code base where we would do something like the above, so I am unsure whether this is a big issue. As long as decorators are directly applied to functions and classes, everything worked fine on our code base. *Our Experience* We have been using icontract in our team for our production code base (~100K LOC) as well as for a couple of open source projects (each <10K LOC) since August 1st 2018 (~ 3 months). In contrast to points raised during the discussions in [1, 2, 3, 4], we did not have issues with readability and modifying contracts. Thus far, we have not encountered problems with variable renames and major code refactorings. We use Pycharm, so it remains open whether this also applies to development environments such as emacs and vim. We confirm that contracts help us improve the documentation, keep the documentation up-to-date, better reason about the function's input and output and catch bugs early and more easily during the unit and integration tests. *Status* The library interface is currently frozen after the version bump to 2.0.0 and is not expected to change in the next six months. All the reported bugs have been fixed and no bug fixes are pending. *Next Steps?* I personally doubt that we are enough people to form a party to push for a change in the language. A standardized library seems to me like a realizable compromise given the state of the discussion on this mail list. Before we organize a collective to write a proposal to standardize the library, I would suggest that a couple of other interested teams adopt icontract, apply it to their code bases and report their experiences on this mail list. I hope that 2-3 reports would be insightful enough to either convince other people that contracts in python are worth standardizing (and highlight missing features yet to be implemented) or provide solid material to discard the endeavor at the current moment. In the meanwhile, it would be of great help if somebody could vet the documentation and the code of icontract. *Authors* The library was mainly written by me (with some help of my colleague Adam Radomski). We discussed the features within the dev team at Parquery (Zurich, Switzerland) as well as in email correspondence with James Lu. [1] https://groups.google.com/forum/#!topic/python-ideas/mvzLsdwGuww [2] https://groups.google.com/forum/#!topic/python-ideas/JtMgpSyODTU [3] https://groups.google.com/forum/#!topic/python-ideas/mvzLsdwGuww [4] https://groups.google.com/forum/#!topic/python-ideas/dmXz_7LH4GI%5B1-25%5D [5] https://groups.google.com/forum/#!topic/python-ideas/c9ntrVuh6WE [6] https://bugs.python.org/issue21217

7 12

Python 3.7 dataclasses attribute order
by Philip Martin Oct. 24, 2018

Oct. 24, 2018

Hi, I just started to use the new dataclasses module. My initial use case boils down to somewhere between a namedtuple and a class where I want a record with a few methods. Mainly, I am trying to build a specific record from various sources, and then have the class handle validation and serialization. One roadblock I have found is that I can't use the field order I define for the class to say write out a csv file if any of the fields have default value. I know this bucks Python's args, kwargs ordering, but I think having the ability define optional arguments and required arguments in any order helps improve the classes intended usability. For instance, imagine "account_description_2" is None for most sources, but must appear before "other_required_field" in a CSV exported file. I think it would be useful to be able to do the following: import csv from datetime import date from dataclasses import dataclass, fields from typing import List OBJECT_SERIALIZERS = {date: date.isoformat} @dataclass class Account: account_id: str account_description: str account_description_2: str = None # INVALID other_required_field: str def serialize(self): for field in fields(self): value = getattr(self, field.name) serializer = OBJECT_SERIALIZERS.get(field.type, None) if serializer: value = serializer(value) yield value @property def field_names(self): return [field.name for field in fields(self)] @classmethod def from_source_a(cls, record): return cls(account_id=record['account_code'], account_description=record['account_name'], other_required_field=record['other_field']) @dataclass class AccountList: accounts: List[Account] @property def field_names(self): return [ field.name for field in fields(fields(self)[0].type.__args__[0]) ] @property def record_field(self): return fields(self)[0].name def to_csv(self, path): with open(path, 'w') as file: self.write_file(file) def write_file(self, file): records_field = self.record_field writer = csv.writer(file) writer.writerow(self.field_names) writer.writerows( record.serialize() for record in getattr(self, records_field) )

3 2

asymcio.TimerHandle.done()
by reg.python＠poti.sk Oct. 24, 2018

Oct. 24, 2018

I'm rewriting an existing python program using a 3rd party async library to standard asyncio. That library allows one to check if a timer created with: timer = loop.call_later(delay, callback) is active, i.e. not cancelled and the scheduled callback not executed yet. I'm missing that functionality in asyncio and would like to propose: TimerHandle.done() = True if cancelled or no longer waiting in the event loop's queue; otherwise False In that program sometimes a state is defined just by the mere existence of an active timer - no state variables, no real callbacks. I find that quite a natural mini-pattern.

1 0

Add closing and iteration to threading.Queue
by Vladimir Filipović Oct. 23, 2018

Oct. 23, 2018

Hi! I originally submitted this as a pull request. Raymond Hettinger suggested it should be given a shakeout in python-ideas first. https://github.com/python/cpython/pull/10018 https://bugs.python.org/issue35034 ------ Briefly: Add a close() method to Queue, which should simplify many common uses of the class and reduce the space for some easy-to-make errors. Also add an __iter__() method which in conjunction with close() would further simplify some common use patterns. ------ At eye-watering length: Apologies in advance for the length of this message. This isn't a PEP in disguise, it's a proposal for a very small, simple and I dare imagine uncontroversial feature. I'm new to contributing to Python and after the BPO/github submission I didn't manage to come up with a better way to present it than this. The issue Code using threading.Queue often needs to coordinate a "work is finished as far as I care" state between the producing and consuming side. Not "work" in the task_done() sense of completion of processing of queue items, "work" in the simpler sense of just passing data through the queue. For example, a producer can be driving the communication by enqueuing e.g. names of files that need to be processed, and once it's enqueued the last filename, it can be useful to inform the consumers that no further names will be coming, so after they've retrieved what's in-flight currently, they don't need to bother waiting for any more. Alternatively, a consumer can be driving the communication, and may need to let the producers know "I'm not interested in any more, so you can stop wasting resources on producing and enqueuing them". Also, a third, coordinating component may need to let both sides know that "Our collective work here is done. Start wrapping it up y'all, but don't drop any items that are still in-flight." In practice it's probably the exception, not the rule, when any piece of code interacting with a Queue _doesn't_ have to either inform another component that its interest in transferring the data has ended, or watch for such information. In the most common case of producer letting consumers know that it's done, this is usually implemented (over and over again) with sentinel objects, which is at best needlessly verbose and at worst error-prone. A recipe for multiple consumers making sure nobody misses the sentinel is not complicated, but neither is it obvious the first time one needs to do it. When a generic sentinel (None or similar) isn't adequate, some component needs to create the sentinel object and communicate it to the others, which complicates code, and especially complicates interfaces between components that are not being developed together (e.g. if one of them is part of a library and expects the library-user code to talk to it through a Queue). In the less common cases where the producers are the ones being notified, there isn't even a typical solution - everything needs to be cooked up from scratch using synchronization primitives. ------ A solution Adding a close() method to the Queue that simply prohibits all further put()'s (with other methods acting appropriately when the queue is closed) would simplify a lot of this in a clean and safe way - for the most obvious example, multi-consumer code would not have to juggle sentinel objects. Adding a further __iter__() method (that would block as necessary, and stop its iteration once the queue is closed and exhausted) would especially simplify many unsophisticated consumers. This is a current fairly ordinary pattern: # Producer: while some_condition: q.put(generate_item()) q.put(sentinel) # Consumer: while True: item = q.get() if item == sentinel: q.put(sentinel) break process(item) (This consumer could be simplified a little with an assignment expression or an iter(q.get, sentinel), but one of those is super new and the other seems little-known in spite of being nearly old enough to vote.) With the close() and __iter__(), this would become: # Producer: with closing(q): while some_condition: q.put(generate_item()) # Consumer: for item in q: process(item) Apart from it being shorter and less error-prone (e.g. if generate_item raises), the implied interface for initializing the two components is also simplified, because there's no sentinel to pass around. More complex consumers that couldn't take advantage of the __iter__() would still benefit from being able to explicitly and readably find out (via exception or querying) that the queue has been closed and exhausted, without juggling the sentinel. I honestly think this small addition would be an unqualified win. And it would not change anything for code that doesn't want to use it. ------ I've got a sample implementation ready for Queue and its children. (Link is at the start of this message. It includes documentation updates too, in case those clarify anything further at this stage.) If this is going in the right direction, I'm happy to do the same for SimpleQueue, but I haven't done it yet. I'm still getting my bearings around the C code base. ------ To immediately answer some of Raymond's initial comments at BPO: This is completely independent from Queue's existing task-tracking protocol. One is about controlling transport, and the other about tracking the completion of processing after transport. I hesitate to call them "orthogonal", but there is no functional overlap between them at all. I keep talking about "common cases" and such weaselly concepts above, and really it's just conjecture based on my own limited experience and informal conversations. BUT, I've also done a survey of the standard library itself. There are four packages that use threading.Queue: concurrent, idlelib, logging, multiprocessing. All except idlelib have at least one piece that would have benefited from a Queue.close() if it had been available when they were being written. I've now had a look at a handful of other languages too: - Java and C# have nothing like this. They basically start from a deque as a collection, and add synchronization features to it; C++ STL doesn't even go that far. None of them do the Python thing where a Queue is a dedicated communication tool that just uses a collection as part of its implementation. - Ruby (to my mild surprise) also does nothing like this. - Clojure does just about the same thing as this proposal, yay. - Go does approximately the same thing, just more finicky about practical usage. The producer can say `close(q)` and the consumers can say `for item := range q { process(item) }` which do exactly the same thing as the proposed Python equivalents, but it has some harsh limitations that are not applicable to Python. (Can't re-close a closed channel, can't query whether a channel is closed outside of retrieving an item). ------ To anticipate a couple more possible questions: - What would this proposal do about multiple producers/consumers needing to jointly decide _when_ to close the queue? Explicitly nothing. The queue's state is either closed or not, and it doesn't care who closed it. It needs to interact correctly with multiple consumers and multiple producers, but once any one piece of code closes it, the correct interaction is acting like a closed queue for everybody. When e.g. multiple producers need to arrange among themselves that the queue be closed only after all of them are done producing, then it's not the queue's job to coordinate _that_. They probably need a Semaphore or a more complex coordinator in charge of the closing. Yes, this too is a non-trivial-to-get-right situation, but trying to solve this one inside the Queue class seems like bloat. - Why not make the Queue a context manager while you're at it? Most closeable classes do it. I don't see that as a clear win in this case. Happy to add it if there's consensus in its favour, of course. I think `with closing(q):` is much more expressive than `with q:` while still brief. The Queue is more versatile in use than files, database cursors and many other resource-type classes, so the meaning of a `with q:` would not be as immediately suggestive as with them. It also often needs to be passed around between creation and initial use (the whole point is that more than one piece of code has access to it) so the common idiom `with Queue() as q:` would be a slightly less natural fit than with resource-like classes. - Should SimpleQueue do the same thing as Queue? Yes, I would propose so and volunteer to implement it. Some details would be adapted to SimpleQueue's existing promises (put() can't fail) but I think this feature is very much in the spirit of SimpleQueue. - Should multiprocessing.Queue do the same thing? I think so, though I'm not proposing it. It already has a close() method, whose meaning is very similar but not identical to (a subset of) the proposed threading.Queue.close's meaning (with resource-management effects not relevant to threading.Queue either way). I'd hope that this "not identical" part would not cause any problems in practice (lots of things aren't identical between those two Queues), but all hoping aside, maybe people more experienced than me can evaluate if that's really the case. I also have no clear idea if the feature would be as valuable, and if the implementation would be as easy and clean as they are with threading.Queue. - Should asyncio.Queue do the same thing? Probably? I'm too unfamiliar with asyncio for my opinion on this to be of value. So I'm not proposing it. ------

7 13

TypeHinting: From variable name to type
by Thomas Güttler Oct. 23, 2018

Oct. 23, 2018

Imaging you are developing in the django context. Everytime you use a variable named "request" or "response" your human brains knows that this is a subclass of django.http.HttpRequest and django.http.HttpResponse. How to give the IDE this knowledge? One solution is the do typehinting everywhere the veriable gets used. But why does the human brain not need this? Because it is intelligent? I would not call this intelligence. There is a simple dictionary in the brain of the developer, which maps: variable-name --> Type And this mapping dict exists once per library. If you are developing in the requests http lib, then there is a different mapping. Then "response" means type requests.Response. Now my idea: Per module and/or per file type hinting from variable name. Maybe a magic docstring in the __init__.py file: """ variable-name-mapping: { request: django.http.HttpRequest, ... } """ This makes the type mapping available for all files in this directory (or sub-directories). What do you think? Disclaimer: For reasons I don't want to explain in detail, I am not allowed to do implement things like this. This is just an idea. I would feel proud and thankfull if someone likes it that much, that he implements it. -- Thomas Guettler http://www.thomas-guettler.de/ I am looking for feedback: https://github.com/guettli/programming-guidelines

10 19

Re: [Python-ideas] [Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals
by Michael Selik Oct. 22, 2018

Oct. 22, 2018

I switched this thread to the python-ideas list, since this is proposing a new feature. On Mon, Oct 22, 2018 at 12:13 PM Sean Harrington <seanharr11(a)gmail.com> wrote: > I contend that multiprocessing.Pool is used most frequently with a single > task. I am proposing a feature that enforces this invariant, optimizes task > memory-footprints & thus serialization time, and preserves the > well-established interface to Pool through subclassing. > We've got a disagreement over interface design. That's more easily discussed with concrete examples. In my own work, I often create a procmap or threadmap function to make that pattern more pleasant. Since you're making the proposal, why not share some examples of using Pool in the manner you'd like? As real a chunk of code as you're able to share publicly (and concisely).

1 0

Python Enhancement Proposal for List methods
by Siva Sukumar Reddy Oct. 22, 2018

Oct. 22, 2018

Hey everyone, I am really new to Python contribution community want to propose below methods for List object. Forgive me if this is not the format to send an email. 1. *list.replace( item_to_be_replaced, new_item )*: which replaces all the occurrences of an element in the list instead of writing a new list comprehension in place. 2. *list.replace( item_to_be_replaced, new_item, number_of_occurrences )*: which replaces the occurrences of an element in the list till specific number of occurrences of that element. The number_of_occurrences can defaulted to 0 which will replace all the occurrences in place. 3. *list.removeall( item_to_be_removed )*: which removes all the occurrences of an element in a list in place. What do you think about these features? Are they PEP-able? Did anyone tried to implement these features before? Please let me know. Thank you, Sukumar

5 6

IPLike objects
by Damla Altun Oct. 21, 2018

Oct. 21, 2018

Hi, Python's ipaddress module is really good for working with network / IP related thing. I would like to use ipaddress.IPv4/v6Address in creating network interfaces. Like a socket but i can't, the internal things and some standard libraries doesn't support them (even socket). >>> server_address = (ipaddress.IPv4Address('127.0.0.1'), 10000) >>> sock.bind(server_address) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: str, bytes or bytearray expected, not IPv4Address Proposal ======== The best solution we've got[0] is implementing something like PathLike[1] for IP Address objects. For PoC[2] we called it IPLike. It is a protocol (some kind of interface) created with Abstract Base Classes and it forces class to implement __ipaddr__ method (like __fspath__ method). The __ipaddr__ method should return string or bytes formed notation of IP address. It is dotted notation in IPv4, long formed notation in IPv6. The __ipaddr__ method is also an identifier for IPLike classes. IPLike interface's subclasshook checks the __ipaddr__ method of given class when issubclass called. >>> class SomethingRepresentIP: ... def __ipaddr__(self) -> str: ... return '127.0.0.1' ... >>> issubclass(SomethingRepresentIP, IPLike) True >>> isinstance(SomethingRepresentIP(), IPLike) True For PoC[2] we implented __ipaddr__ method to ipaddress._BaseAddress class. It is parent class of IPv4/v6Adress. >>> issubclass(IPv4Address, IPLike) True >>> IPv4Address('127.0.0.1').__ipaddr__() '127.0.0.1' Like os.fspath we need a general-purpose ip-getter. So we implemented ipaddress.get_ipaddr(address) function. When you give it an valid IP like object it returns the value of __ipaddr__ method. >>> get_ipaddr(IPv4Address('127.0.0.1')) '127.0.0.1' When you give it string or bytes or integer object it calls ipaddr.ip_address(address) and re-calls itself with the value it got from ip_address. It helps get_ipaddr to convert integer-formed / byte-formed IPs to string formed IPs with a valid notation. >>> get_ipaddr('127.0.0.1') '127.0.0.1' >>> get_ipaddr(3232235521) '192.168.0.1' >>> get_ipaddr(b'\xC0\xA8\x00\x01') '192.168.0.1' If you give any type else it raises TypeError. Also if you give it a IP like object (an object that has __ipaddr__ method), but the object's __ipaddr__ method returns something that is not string/bytes it raises TypeError. For PoC[2] i added socket to support IP like objects. >>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> server_address = (SomethingRepresentIP(), 10000) >>> sock.bind(server_address) >>> sock.listen(1) >>> conn, client = sock.accept() >>> conn <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 10000), raddr=('127.0.0.1', 49452)> I didn't implement a C API for ipaddress.get_ipaddr. The function is too small for creating a new module, new header and the ipaddress library currently has no C-side module. If needed i can write a function inside the socketmodule.c . Proof-of-Concept (Implementation) ================================= The implementation can be found on github (DamlaAltun/cpython[2]) We have a Abstract Base Class for IP Address protocol called IPLike. (ipaddress.py[3]) class IPLike(abc.ABC): """ Abstract base class for IP (address) like objects. """ @abc.abstractmethod def __ipaddr__(self): """ If it is a IPv4 return in the dotted form, If it is a IPv6 return in the long form. """ raise NotImplementedError @classmethod def __subclasshook__(cls, subclass): return hasattr(subclass, '__ipaddr__') It has a abstract method __ipaddr__ and a subclasshook. The __ipaddr__ method returns ip representation of object. The subclasshook checks for __ipaddr__ method. Then there is a ip-getter called get_ipaddr (ipaddress.py[4]) def get_ipaddr(address): """Return the representation of a ip address like object. If bytes or string or int passed in, get_ipaddr converts it to a IPv4/v6 Address and calls it self again for getting, IP address of IPv4/v6 object. If a IPLike passed it uses IPLike interface to get the value. """ if isinstance(address, (str, bytes, int)): return get_ipaddr(ip_address(address)) addr_type = type(address) try: ip_addr = address.__ipaddr__() except AttributeError: if hasattr(address, '__ipaddr__'): raise else: raise TypeError("expected str, bytes or ipaddress.IPLike object, " "not {}".format(addr_type.__name__)) if isinstance(ip_addr, (str, bytes)): return ip_addr else: raise TypeError("expected {}.__ipaddr__() to return str or bytes, " "not {}".format(addr_type.__name__, type(addr_type).__name__)) It takes one parameter, address. The address might be string, integer, bytes or ipaddress.IPLike If it is string or bytes or integer the get_ipaddr calls ipaddress.ip_address for getting IPv4/v6Address object. The purpose of that is converting bytes/int IP addresses to valid formed string IP addresses. And calls it self again with result of ipaddress.ip_address It tries to get address' __ipaddr__ method, if it can successfully perform it checks the return value of __ipaddr__ method. If return value is string or bytes it returns the ip address. If it is not it raises a TypeError. In the C-Side we can check via _PyObject_LookupSpecial. (socketmodule.c[5]) idna_converter(PyObject *obj, struct maybe_idna *data) { ... _Py_IDENTIFIER(__ipaddr__); ... func = _PyObject_LookupSpecial(obj, &PyId___ipaddr__); if (func != NULL) { obj = PyObject_CallFunctionObjArgs(func, NULL); Py_DECREF(func); } } Thanks for your attention, Damla [0] https://github.com/python/cpython/pull/9956#pullrequestreview-166756568 [1] https://www.python.org/dev/peps/pep-0519 [2] https://github.com/DamlaAltun/cpython/tree/iplike-object [3] https://github.com/DamlaAltun/cpython/blob/iplike-object/Lib/ipaddress.py#L… [4] https://github.com/DamlaAltun/cpython/blob/iplike-object/Lib/ipaddress.py#L… [5] https://github.com/DamlaAltun/cpython/blob/iplike-object/Modules/socketmodu…

2 2

Re: [Python-ideas] Python-ideas Digest, Vol 143, Issue 73
by Bartłomiej Bartnicki Oct. 19, 2018

Oct. 19, 2018

I agree with Ivan. This feater is stands in contradiction with second Python Zen principle: "Explicit is better than implicit.". Isn't it? On Fri, Oct 19, 2018 at 11:41 AM <python-ideas-request(a)python.org> wrote: > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > > Today's Topics: > > 1. TypeHinting: From variable name to type (Thomas G?ttler) > 2. Re: TypeHinting: From variable name to type (Anders Hovm?ller) > 3. Re: TypeHinting: From variable name to type (Robert Vanden Eynde) > 4. Re: TypeHinting: From variable name to type (Thomas G?ttler) > 5. Re: TypeHinting: From variable name to type (Ivan Levkivskyi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 19 Oct 2018 09:18:02 +0200 > From: Thomas G?ttler <guettliml(a)thomas-guettler.de> > To: python-ideas(a)python.org > Subject: [Python-ideas] TypeHinting: From variable name to type > Message-ID: <2d9d1825-2619-e4a0-1b19-6a614666816a(a)thomas-guettler.de> > Content-Type: text/plain; charset=utf-8; format=flowed > > Imaging you are developing in the django context. > > Everytime you use a variable named "request" or "response" your human > brains > knows that this is a subclass of django.http.HttpRequest and > django.http.HttpResponse. > > How to give the IDE this knowledge? > > One solution is the do typehinting everywhere the veriable gets used. > > But why does the human brain not need this? > > Because it is intelligent? > > I would not call this intelligence. There is a simple > dictionary in the brain of the developer, which maps: > > variable-name --> Type > > And this mapping dict exists once per library. > > If you are developing in the requests http lib, then > there is a different mapping. Then "response" means > type requests.Response. > > > Now my idea: Per module and/or per file type hinting from variable name. > > Maybe a magic docstring in the __init__.py file: > > """ > variable-name-mapping: > { > request: django.http.HttpRequest, > ... > } > """ > > This makes the type mapping available for all files in this directory (or > sub-directories). > > What do you think? > > > > > Disclaimer: For reasons I don't want to explain in detail, I am not > allowed to > do implement things like this. This is just an idea. I would feel proud > and thankfull > if someone likes it that much, that he implements it. > > > > > -- > Thomas Guettler http://www.thomas-guettler.de/ > I am looking for feedback: > https://github.com/guettli/programming-guidelines > > > ------------------------------ > > Message: 2 > Date: Fri, 19 Oct 2018 09:29:38 +0200 > From: Anders Hovm?ller <boxed(a)killingar.net> > To: Thomas G?ttler <guettliml(a)thomas-guettler.de> > Cc: python-ideas(a)python.org > Subject: Re: [Python-ideas] TypeHinting: From variable name to type > Message-ID: <3A295990-52A3-4347-865E-096E9126C6C7(a)killingar.net> > Content-Type: text/plain; charset=utf-8 > > > > > On 19 Oct 2018, at 09:18, Thomas G?ttler <guettliml(a)thomas-guettler.de> > wrote: > > > > Imaging you are developing in the django context. > > > > Everytime you use a variable named "request" or "response" your human > brains > > knows that this is a subclass of django.http.HttpRequest and > django.http.HttpResponse. > > > > How to give the IDE this knowledge? > > > > One solution is the do typehinting everywhere the veriable gets used. > > > > But why does the human brain not need this? > > > > Because it is intelligent? > > > > I would not call this intelligence. There is a simple > > dictionary in the brain of the developer, which maps: > > > > variable-name --> Type > > > > And this mapping dict exists once per library. > > > > If you are developing in the requests http lib, then > > there is a different mapping. Then "response" means > > type requests.Response. > > > > > > Now my idea: Per module and/or per file type hinting from variable name. > > > > Maybe a magic docstring in the __init__.py file: > > > > """ > > variable-name-mapping: > > { > > request: django.http.HttpRequest, > > ... > > } > > """ > > > > This makes the type mapping available for all files in this directory > (or sub-directories). > > > > What do you think? > > > I tried to implement this in mypy quite recently actually, but gave up for > various reasons. > > I am very much +1 on this. This would be a huge boon to preexisting code > bases. > > / Anders > > > > > ------------------------------ > > Message: 3 > Date: Fri, 19 Oct 2018 10:22:08 +0200 > From: Robert Vanden Eynde <robertve92(a)gmail.com> > To: Thomas G?ttler <guettliml(a)thomas-guettler.de> > Cc: python-ideas <python-ideas(a)python.org> > Subject: Re: [Python-ideas] TypeHinting: From variable name to type > Message-ID: > < > CA+msPNkhxXxCWkB4f_YbeR3oJQLNBWLGzvsOfouW+BQ7nonFyA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > That would be a feature in my "python dialect" future library. Taking one > valid python program and producing another python program : > > def f(request): > return HttpResponse('ok') > > ? > > def f(request: HttpRequest): > return HttpResponse('ok') > > The dialect options would include "all top functions in this module" for > example, and parameters about the naming convention (explicit list of > conversion or rules to create the mapping). > > Le ven. 19 oct. 2018 ? 09:18, Thomas G?ttler <guettliml(a)thomas-guettler.de > > > a ?crit : > > > Imaging you are developing in the django context. > > > > Everytime you use a variable named "request" or "response" your human > > brains > > knows that this is a subclass of django.http.HttpRequest and > > django.http.HttpResponse. > > > > How to give the IDE this knowledge? > > > > One solution is the do typehinting everywhere the veriable gets used. > > > > But why does the human brain not need this? > > > > Because it is intelligent? > > > > I would not call this intelligence. There is a simple > > dictionary in the brain of the developer, which maps: > > > > variable-name --> Type > > > > And this mapping dict exists once per library. > > > > If you are developing in the requests http lib, then > > there is a different mapping. Then "response" means > > type requests.Response. > > > > > > Now my idea: Per module and/or per file type hinting from variable name. > > > > Maybe a magic docstring in the __init__.py file: > > > > """ > > variable-name-mapping: > > { > > request: django.http.HttpRequest, > > ... > > } > > """ > > > > This makes the type mapping available for all files in this directory (or > > sub-directories). > > > > What do you think? > > > > > > > > > > Disclaimer: For reasons I don't want to explain in detail, I am not > > allowed to > > do implement things like this. This is just an idea. I would feel proud > > and thankfull > > if someone likes it that much, that he implements it. > > > > > > > > > > -- > > Thomas Guettler http://www.thomas-guettler.de/ > > I am looking for feedback: > > https://github.com/guettli/programming-guidelines > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas(a)python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > >

1 0