Mailman 3 June 2018 - Python-ideas

Re: [Python-ideas] [Distutils] Pypi private repo's
by Nick Coghlan Oct. 28, 2018

Oct. 28, 2018

On 5 April 2018 at 07:58, Jannis Gebauer <ja.geb(a)me.com> wrote: > What if there was some kind of “blessed” entity that runs these services and puts the majority of the revenue into a fund that funds development on PyPi (maybe trough the PSF)? Having a wholly owned for-profit subsidiary that provides commercial services as a revenue raising mechanism is certainly one way to approach something like this without alienating sponsors or tax authorities (although it may still alienate the … [View More]vendors of now competing services). It would require a big time commitment on the PSF side to get everything set up though, as well as interest from key folks in joining what would essentially be a single-language-focused start up in an already crowded cross-language developer tools marketplace. When the PSF as a whole is still operating with only a handful of full or part time employees, it's far from clear that setting something like that up would be the most effective possible use of their time and energy. At a more basic level, that kind of arrangement technically doesn't require anyone's blessing, it could be as straightforward as downstream tooling vendors signing up as PSF sponsors and saying "please allocate our sponsorship contribution to the Packaging WG's budget so that PyPI keeps operating well and the PyPA tooling keeps improving, increasing the level of demand for our commercial Python repository management services". Historically that wouldn't have helped much, since the PSF itself has struggled with effective project management (for a variety of reasons), but one of the things I think the success of the MOSS grant has shown is the significant strides that the PSF has made in budget management in recent years, such that if funding is made available, it can and will be spent effectively. Cheers, Nick. P.S. PyPA contributors are also free agents in their own right, so folks offering Python-centric developer workflow management tools or features may decide that it's worth their while to invest more directly in smoothing out some of the rough edges that currently still exist. It's a mercenary way of looking at things, but in many cases, it is *absolutely* possible to pay for the time and attention of existing contributors, and if you can persuade them that your proposals are reasonable, they'll often have an easier time than most convincing other community contributors that it's a good way to go :) -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia [View Less]

3 5

Asynchronous exception handling around with/try statement borders
by Erik Bray Sept. 24, 2018

Sept. 24, 2018

Hi folks, I normally wouldn't bring something like this up here, except I think that there is possibility of something to be done--a language documentation clarification if nothing else, though possibly an actual code change as well. I've been having an argument with a colleague over the last couple days over the proper way order of statements when setting up a try/finally to perform cleanup of some action. On some level we're both being stubborn I think, and I'm not looking for resolution … [View More]as to who's right/wrong or I wouldn't bring it to this list in the first place. The original argument was over setting and later restoring os.environ, but we ended up arguing over threading.Lock.acquire/release which I think is a more interesting example of the problem, and he did raise a good point that I do want to bring up. </prologue> My colleague's contention is that given lock = threading.Lock() this is simply *wrong*: lock.acquire() try: do_something() finally: lock.release() whereas this is okay: with lock: do_something() Ignoring other details of how threading.Lock is actually implemented, assuming that Lock.__enter__ calls acquire() and Lock.__exit__ calls release() then as far as I've known ever since Python 2.5 first came out these two examples are semantically *equivalent*, and I can't find any way of reading PEP 343 or the Python language reference that would suggest otherwise. However, there *is* a difference, and has to do with how signals are handled, particularly w.r.t. context managers implemented in C (hence we are talking CPython specifically): If Lock.__enter__ is a pure Python method (even if it maybe calls some C methods), and a SIGINT is handled during execution of that method, then in almost all cases a KeyboardInterrupt exception will be raised from within Lock.__enter__--this means the suite under the with: statement is never evaluated, and Lock.__exit__ is never called. You can be fairly sure the KeyboardInterrupt will be raised from somewhere within a pure Python Lock.__enter__ because there will usually be at least one remaining opcode to be evaluated, such as RETURN_VALUE. Because of how delayed execution of signal handlers is implemented in the pyeval main loop, this means the signal handler for SIGINT will be called *before* RETURN_VALUE, resulting in the KeyboardInterrupt exception being raised. Standard stuff. However, if Lock.__enter__ is a PyCFunction things are quite different. If you look at how the SETUP_WITH opcode is implemented, it first calls the __enter__ method with _PyObjet_CallNoArg. If this returns NULL (i.e. an exception occurred in __enter__) then "goto error" is executed and the exception is raised. However if it returns non-NULL the finally block is set up with PyFrame_BlockSetup and execution proceeds to the next opcode. At this point a potentially waiting SIGINT is handled, resulting in KeyboardInterrupt being raised while inside the with statement's suite, and finally block, and hence Lock.__exit__ are entered. Long story short, because Lock.__enter__ is a C function, assuming that it succeeds normally then with lock: do_something() always guarantees that Lock.__exit__ will be called if a SIGINT was handled inside Lock.__enter__, whereas with lock.acquire() try: ... finally: lock.release() there is at last a small possibility that the SIGINT handler is called after the CALL_FUNCTION op but before the try/finally block is entered (e.g. before executing POP_TOP or SETUP_FINALLY). So the end result is that the lock is held and never released after the KeyboardInterrupt (whether or not it's handled somehow). Whereas, again, if Lock.__enter__ is a pure Python function there's less likely to be any difference (though I don't think the possibility can be ruled out entirely). At the very least I think this quirk of CPython should be mentioned somewhere (since in all other cases the semantic meaning of the "with:" statement is clear). However, I think it might be possible to gain more consistency between these cases if pending signals are checked/handled after any direct call to PyCFunction from within the ceval loop. Sorry for the tl;dr; any thoughts? [View Less]

7 15

Positional-only parameters
by Victor Stinner Sept. 11, 2018

Sept. 11, 2018

Hi, For technical reasons, many functions of the Python standard libraries implemented in C have positional-only parameters. Example: ------- $ ./python Python 3.7.0a0 (default, Feb 25 2017, 04:30:32) >>> help(str.replace) replace(self, old, new, count=-1, /) # <== notice "/" at the end ... >>> "a".replace("x", "y") # ok 'a' >>> "a".replace(old="x", new="y") # ERR! TypeError: replace() takes at least 2 arguments (0 given) ------- When converting the … [View More]

19 27

Fwd: Replacing Infinite while Loops with an Iterator: async edition
by jab＠math.brown.edu Aug. 24, 2018

Aug. 24, 2018

On Jun 23, 2018, at 21:11, Nathaniel Smith <n...(a)pobox.com> wrote: > He's asking for an async version of the 'iter' builtin, presumably > something like: > async def aiter(async_callable, sentinel): > while True: > value = await async_callable() > if value == sentinel: > break > yield value > -n Yes, exactly (thanks, Nathaniel). Wouldn't that be a useful built-in? (Greg, I too would be surprised if this were the first … [View More]

3 4

Allow a group by operation for dict comprehension
by Nicolas Rolin July 18, 2018

July 18, 2018

Hi, I use list and dict comprehension a lot, and a problem I often have is to do the equivalent of a group_by operation (to use sql terminology). For example if I have a list of tuples (student, school) and I want to have the list of students by school the only option I'm left with is to write student_by_school = defaultdict(list) for student, school in student_school_list: student_by_school[school].append(student) What I would expect would be a syntax with comprehension … [View More]allowing me to write something along the lines of: student_by_school = {group_by(school): student for school, student in student_school_list} or any other syntax that allows me to regroup items from an iterable. Small FAQ: Q: Why include something in comprehensions when you can do it in a small number of lines ? A: A really appreciable part of the list and dict comprehension is the fact that it allows the developer to be really explicit about what he wants to do at a given line. If you see a comprehension, you know that the developer wanted to have an iterable and not have any side effect other than depleting the iterator (if he respects reasonable code guidelines). Initializing an object and doing a for loop to construct it is both too long and not explicit enough about what is intended. It should be reserved for intrinsically complex operations, not one of the base operation one can want to do with lists and dicts. Q: Why group by in particular ? A: If we take SQL queries (https://en.wikipedia.org/wiki/SQL_syntax#Queries) as a reasonable way of seeing how people need to manipulate data on a day-to-day basis, we can see that dict comprehensions already covers most of the base operations, the only missing operations being group by and having. Q: Why not use it on list with syntax such as student_by_school = [ school, student for school, student in student_school_list group by school ] ? A: It would create either a discrepancy with iterators or a perhaps misleading semantic (the one from itertools.groupby, which requires the iterable to be sorted in order to be useful). Having the option do do it with a dict remove any ambiguity and should be enough to cover most "group by" applications. Examples: edible_list = [('fruit', 'orange'), ('meat', 'eggs'), ('meat', 'spam'), ('fruit', 'apple'), ('vegetable', 'fennel'), ('fruit', 'pineapple'), ('fruit', 'pineapple'), ('vegetable', 'carrot')] edible_list_by_food_type = {group_by(food_type): edible for food_type, edible in edible_list} print(edible_list_by_food_type) {'fruit': ['orange', 'pineapple'], 'meat': ['eggs', 'spam'], 'vegetable': ['fennel', 'carrot']} bank_transactions = [200.0, -357.0, -9.99, -15.6, 4320.0, -1200.0] splited_bank_transactions = {group_by('credit' if amount > 0 else 'debit'): amount for amount in bank_transactions} print(splited_bank_transactions) {'credit': [200.0, 4320.0], 'debit': [-357.0, -9.99, -15.6, -1200.0]} -- Nicolas Rolin [View Less]

20 84

Add a __cite__ method for scientific packages
by Andrei Kucharavy July 5, 2018

July 5, 2018

Over the last 10 years, Python has slowly inched towards becoming the most popular scientific computing language, beating or seriously challenging Matlab, R, Mathematica and many specialized languages (S, SAS, ...) in numerous applications. A large part of this growth is driven by amazing community packages, such as numpy, scipy, scikits-learn, scikits-image, seaborn or pandas, just to name a few. Development of such packages represents a significant time investment by people working in … [View More]

18 28

Should nested classes in an Enum be Enum members?
by Ethan Furman July 1, 2018

July 1, 2018

Consider the following Enum definition: class Color(Enum): RED = 1 GREEN = 2 BLUE = 3 @property def lower(self): return self.name.lower() def spam(self): return "I like %s eggs and spam!" % self.lower class SomeClass: pass Which of the above Color attributes are enums, and which aren't? . . . Answer: - RED, GREEN, and BLUE are members - lower and spam() are not - SomeClass /is/ a member (… [View More]

10 15

Fwd: collections.Counter should implement fromkeys
by Tim Peters June 30, 2018

June 30, 2018

Reposting because the original got bounced from Google Groups. ---------- Forwarded message --------- From: Tim Peters <tim.peters(a)gmail.com> Date: Fri, Jun 29, 2018 at 5:54 PM Subject: Re: [Python-ideas] collections.Counter should implement fromkeys To: <abedillon(a)gmail.com> Cc: python-ideas <python-ideas(a)googlegroups.com> [Abe Dillon <abedillon(a)gmail.com>] > ... > I'm using the copy-constructor because I know Counter is a subclass of dict. > I'm … [View More]

4 6

Re: [Python-ideas] collections.Counter should implement fromkeys
by Abe Dillon June 30, 2018

June 30, 2018

[Michael Selik] > You might be pursuing a local optimum of obviousness. If you step back > from the idea of "initialize the counts with all interesting keys" and > looping over them, you might find a better overall solution. > That's a distinct possibility, but this is far from the first time I've wanted for a better way to initialize a Counter. It's just a simple example for which there are many many alternative approaches. If' you're interested, I have a more complete … [View More]

1 0

Re: [Python-ideas] collections.Counter should implement fromkeys
by Abe Dillon June 29, 2018

June 29, 2018

[Michael Selik] > You need an iterable of the keys you're interested in to pass to the > hypothetical ``fromkeys`` method. Why not iterate over that same iterable > paired with zeros instead of passing it into ``fromkeys``? > Because, as in my original example code; the values could be zero or they could be more, I just want to make sure the keys are in the counter when I iterate. I'm not having any trouble finding a work around. I'm having trouble understanding why I need … [View More]

1 0