Frequently, while globbing, one needs to work with multiple extensions. I’d
like to propose for fnmatch.filter to handle a tuple of patterns (while
preserving the single str argument functionality, alas str.endswith), as a
first step for glob.i?glob to accept multiple patterns as well.
Here is the implementation I came up with:
https://github.com/python/cpython/compare/master...andresdelfino:fnmatch-mu…
If this is deemed reasonable, I’ll write tests and documentation updates.
Any opinion?
The proposed implementation of dataclasses prevents defining fields with
defaults before fields without defaults. This can create limitations on
logical grouping of fields and on inheritance.
Take, for example, the case:
@dataclass
class Foo:
some_default: dict = field(default_factory=dict)
@dataclass
class Bar(Foo):
other_field: int
this results in the error:
5 @dataclass
----> 6 class Bar(Foo):
7 other_field: int
8
~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py
in dataclass(_cls, init, repr, eq, order, hash, frozen)
751
752 # We're called as @dataclass, with a class.
--> 753 return wrap(_cls)
754
755
~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py
in wrap(cls)
743
744 def wrap(cls):
--> 745 return _process_class(cls, repr, eq, order, hash, init,
frozen)
746
747 # See if we're being called as @dataclass or @dataclass().
~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py
in _process_class(cls, repr, eq, order, hash, init, frozen)
675 # in __init__. Use "self" if
possible.
676 '__dataclass_self__' if 'self' in
fields
--> 677 else 'self',
678 ))
679 if repr:
~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py
in _init_fn(fields, frozen, has_post_init, self_name)
422 seen_default = True
423 elif seen_default:
--> 424 raise TypeError(f'non-default argument {f.name!r} '
425 'follows default argument')
426
TypeError: non-default argument 'other_field' follows default argument
I understand that this is a limitation of positional arguments because the
effective __init__ signature is:
def __init__(self, some_default: dict = <something>, other_field: int):
However, keyword only arguments allow an entirely reasonable solution to
this problem:
def __init__(self, *, some_default: dict = <something>, other_field: int):
And have the added benefit of making the fields in the __init__ call
entirely explicit.
So, I propose the addition of a keyword_only flag to the @dataclass
decorator that renders the __init__ method using keyword only arguments:
@dataclass(keyword_only=True)
class Bar(Foo):
other_field: int
--George Leslie-Waksman
Forgive me if this idea has been discussed before, I searched the mailing lists, the CPython repo, and the issue tracker and was unable to find anything.
I have found myself a few times in a position where I have a repeated argument that uses the `append` action, along with some convenience arguments that append a specific const to that same dest (eg: `--filter-x` being made equivalent to `--filter x` via `append_const`). This is particularly useful in cli apps that expose some kind of powerful-but-verbose filtering capability, while also providing shorter aliases for common invocations. I'm sure there are other use cases, but this is the one I'm most familiar with.
The natural extension to this filtering idea are convenience args that set two const values (eg: `--filter x --filter y` being equivalent to `--filter-x-y`), but there is no `extend_const` action to enable this.
While this is possible (and rather straight forward) to add via a custom action, I feel like this should be a built-in action instead. `append` has `append_const`, it seems intuitive and reasonable to expect `extend` to have `extend_const` too (my anecdotal experience the first time I came across this need was that I simply tried using `extend_const` without checking the docs, assuming it already existed).
Please see this gist for a working example that may help explain the idea and intended use case more clearly: https://gist.github.com/roganartu/7c2ec129d868ecda95acfbd655ef0ab2
Hi Ilya,
I'm not sure that this mailing list (Python-Dev) is the right place for
this discussion, I think that Python-Ideas (CCed) is the correct place.
For the benefit of Python-Ideas, I have left your entire post below, to
establish context.
[Ilya]
> I needed reversed(enumerate(x: list)) in my code, and have discovered
> that it wound't work. This is disappointing because operation is well
> defined.
It isn't really well-defined, since enumerate can operate on infinite
iterators, and you cannot reverse an infinite stream. Consider:
def values():
while True:
yield random.random()
a, b = reversed(enumerate(values())
What should the first pair of (a, b) be?
However, having said that, I think that your idea is not unreasonable.
`enumerate(it)` in the most general case isn't reversable, but if `it`
is reversable and sized, there's no reason why `enumerate(it)` shouldn't
be too.
My personal opinion is that this is a fairly obvious and straightforward
enhancement, one which (hopefully!) shouldn't require much, if any,
debate. I don't think we need a new class for this, I think enhancing
enumerate to be reversable if its underlying iterator is reversable
makes good sense.
But if you can show some concrete use-cases, especially one or two from
the standard library, that would help your case. Or some other languages
which offer this functionality as standard.
On the other hand, I think that there is a fairly lightweight work
around. Define a helper function:
def countdown(n):
while True:
yield n
n -= 1
then call it like this:
# reversed(enumerate(seq))
zip(countdown(len(seq)-1), reversed(seq)))
So it isn't terribly hard to work around this. But I agree that it would
be nice if enumerate encapsulated this for the caller.
One potentially serious question: what should `enumerate.__reversed__`
do when given a starting value?
reversed(enumerate('abc', 1))
Should that yield...?
# treat the start value as a start value
(1, 'c'), (0, 'b'), (-1, 'a')
# treat the start value as an end value
(3, 'c'), (2, 'b'), (1, 'a')
Something else?
My preference would be to treat the starting value as an ending value.
Steven
On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:
> Hi,
>
> I needed reversed(enumerate(x: list)) in my code, and have discovered that
> it wound't work. This is disappointing because operation is well defined.
> It is also well defined for str type, range, and - in principle, but not
> yet in practice - on dictionary iterators - keys(), values(), items() as
> dictionaries are ordered now.
> It would also be well defined on any user type implementing __iter__,
> __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors.
>
> That's plenty of usecases, therefore I guess it would be quite useful to
> avoid hacky / inefficient solutions like described here:
> https://code.activestate.com/lists/python-list/706205/.
>
> If deemed useful, I would be interested in implementing this, maybe
> together with __reversed__ on dict keys, values, items.
>
> Best Regards,
> --
> Ilya Kamen
>
> -----------
> p.s.
>
> *Sketch* of what I am proposing:
>
> class reversible_enumerate:
>
> def __init__(self, iterable):
> self.iterable = iterable
> self.ctr = 0
>
> def __iter__(self):
> for e in self.iterable:
> yield self.ctr, e
> self.ctr += 1
>
> def __reversed__(self):
> try:
> ri = reversed(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> try:
> l = len(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> indexes = range(l-1, -1, -1)
> for i, e in zip(indexes, ri):
> yield i, e
>
> for i, c in reversed(reversible_enumerate("Hello World")):
> print(i, c)
>
> for i, c in reversed(reversible_enumerate([11, 22, 33])):
>
> print(i, c)
> _______________________________________________
> Python-Dev mailing list -- python-dev(a)python.org
> To unsubscribe send an email to python-dev-leave(a)python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NDDKDUD…
> Code of Conduct: http://python.org/psf/codeofconduct/
Consider the following example:
import unittest
def foo():
for x in [1, 2, 'oops', 4]:
print(x + 100)
class TestFoo(unittest.TestCase):
def test_foo(self):
self.assertIs(foo(), None)
if __name__ == '__main__':
unittest.main()
If we were calling `foo` directly we could enter post-mortem debugging via `python -m pdb test.py`.
However since `foo` is wrapped in a test case, `unittest` eats the exception and thus prevents post-mortem debugging. `--failfast` doesn't help, the exception is still swallowed.
Since I am not aware of a solution that enables post-mortem debugging in such a case (without modifying the test scripts, please correct me if one exists), I propose adding a command-line option to `unittest` for [running test cases in debug mode](https://docs.python.org/3/library/unittest.html#unittest.TestCase.deb… so that post-mortem debugging can be used.
P.S.: There is also [this SO question](https://stackoverflow.com/q/4398967/3767239) on a similar topic.
Hi all,
I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !!
There's a whole matrix of these and I'm wondering why the matrix is
currently sparse rather than implementing them all. Or rather, why we
can't stack them as:
class foo(object):
@classmethod
@property
def bar(cls, ...):
...
Essentially the permutation are, I think:
{'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable
attribute}.
concreteness
implicit first arg
type
name
comments
{unadorned}
{unadorned}
method
def foo():
exists now
{unadorned} {unadorned} property
@property
exists now
{unadorned} {unadorned} non-callable attribute
x = 2
exists now
{unadorned} static
method @staticmethod
exists now
{unadorned} static property @staticproperty
proposing
{unadorned} static non-callable attribute {degenerate case -
variables don't have arguments}
unnecessary
{unadorned} class
method @classmethod
exists now
{unadorned} class property @classproperty or @classmethod;@property
proposing
{unadorned} class non-callable attribute {degenerate case - variables
don't have arguments}
unnecessary
abc.abstract {unadorned} method @abc.abstractmethod
exists now
abc.abstract {unadorned} property @abc.abstractproperty
exists now
abc.abstract {unadorned} non-callable attribute
@abc.abstractattribute or @abc.abstract;@attribute
proposing
abc.abstract static method @abc.abstractstaticmethod
exists now
abc.abstract static property @abc.staticproperty
proposing
abc.abstract static non-callable attribute {degenerate case -
variables don't have arguments} unnecessary
abc.abstract class method @abc.abstractclassmethod
exists now
abc.abstract class property @abc.abstractclassproperty
proposing
abc.abstract class non-callable attribute {degenerate case -
variables don't have arguments} unnecessary
I think the meanings of the new ones are pretty straightforward, but in
case they are not...
@staticproperty - like @property only without an implicit first
argument. Allows the property to be called directly from the class
without requiring a throw-away instance.
@classproperty - like @property, only the implicit first argument to the
method is the class. Allows the property to be called directly from the
class without requiring a throw-away instance.
@abc.abstractattribute - a simple, non-callable variable that must be
overridden in subclasses
@abc.abstractstaticproperty - like @abc.abstractproperty only for
@staticproperty
@abc.abstractclassproperty - like @abc.abstractproperty only for
@classproperty
--rich
In Python 3.10 we will no longer be burdened by the old parser (though 3rd
party tooling needs to catch up).
One thing that the PEG parser makes possible in about 20 lines of code is
something not entirely different from the old print statement. I have a
prototype:
Python 3.10.0a0 (heads/print-statement-dirty:5ed19fcc1a, Jun 9 2020,
16:31:17)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Cannot read termcap database;
using dumb terminal settings.
>>> print 2+2
4
>>> print "hello world"
hello world
>>> print "hello", input("Name:")
Name:Guido
hello Guido
>>> print 1, 2, 3, sep=", "
1, 2, 3
>>>
But wait, there's more! The same syntax will make it possible to call *any*
function:
>>> len "abc"
3
>>>
Or any method:
>>> import sys
>>> sys.getrefcount "abc"
24
>>>
Really, *any* method:
>>> class C:
... def foo(self, arg): print arg
...
>>> C().foo 2+2
4
>>>
There are downsides too, though. For example, you can't call a method
without arguments:
>>> print
<built-in function print>
>>>
Worse, the first argument cannot start with a parenthesis or bracket:
>>> print (1, 2, 3)
1 2 3
>>> C().foo (1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: C.foo() takes 2 positional arguments but 4 were given
>>> print (2+2), 42
4
(None, 42)
>>> C().foo [0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'method' object is not subscriptable
>>>
No, it's not April 1st. I am seriously proposing this (but I'll withdraw it
if the response is a resounding "boo, hiss"). After all, we currently have
a bunch of complexity in the parser just to give a helpful error message to
people used to Python 2's print statement:
>>> print 1, 2, 3
File "<stdin>", line 1
print 1, 2, 3
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1,
2, 3)?
>>>
And IIRC there have been a number of aborted attempts at syntactic hacks to
allow people to call functions (like print) without parentheses, although
(I think) none of them made it into a PEP. The PEG parser makes this much
simpler, because it can simply backtrack -- by placing the grammar rule for
this syntax (tentatively called "call statement") last in the list of
alternatives for "small statement" we ensure that everything that's a valid
expression statement (including print() calls) is still an expression
statement with exactly the same meaning, while still allowing
parameter-less function calls, without lexical hacks. (There is no code in
my prototype that checks for a space after 'print' -- it just checks that
there's a name, number or string following a name, which is never legal
syntax.)
One possible extension I didn't pursue (yet -- dare me!) is to allow
parameter-less calls inside other expressions. For example, my prototype
does not support things like this:
>>> a = (len "abc")
File "<stdin>", line 1
a = (len "abc")
^
SyntaxError: invalid syntax
>>>
I think that strikes a reasonable balance between usability and reduced
detection of common errors.
I could also dial it back a bit, e.g. maybe it's too much to allow 'C().foo
x' and we should only allow dotted names (sufficient to access functions in
imported modules and method calls on variables). Or maybe we should only
allow simple names (allowing 'len x' but disallowing 'sys.getrefcount x'.
Or maybe we should really only bring back print statements.
I believe there are some other languages that support a similar grammar
(Ruby? R? Raku?) but I haven't investigated.
Thoughts?
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>
On Wed, 22 Jul 2020 at 15:35, Antoine Pitrou <solipsis(a)pitrou.net> wrote:
> The deltas also look small for a micro-benchmark. I certainly don't
> think this is a sufficient reason to add a new datatype to Python.
>
I think some of the optimizations can be experimented with dict itself.
Deltas seem to be small, because I divide them by the number of timeit
runs. I think that a 30-45% gain in constructor speed is not something
small. The benchmarks in question have the Name that starts with
"constructor".
PS: sorry for double mail, I sent this message to only you by mistake.
On Wed, 22 Jul 2020 at 18:26, Guido van Rossum <guido(a)python.org> wrote:
> Did you study PEP 416 (frozendict) and PEP 603 (frozenmap)?
>
Yes. About frozenmap, at the very start I added to the benchmarks also
immutables.Map, but I removed it when I realized that it was slower than
frozendict in every bench. Maybe this is because immutables.Map is a C
extension and not a builtin type.
About PEP 416, yes, I tried to follow it. Indeed hash is calculated using
the strategy described in the PEP. I also take a look to frozenset and
tuple code.
Frankly I must admit that the rejection of PEP 416 was a good idea. Indeed
I started to implement an immutable dict only for a symmetry reason, and
because I am somewhat fascinated by functional programming and immutables,
without a rational reason.
I suppose I would have given up and admitted that a frozendict is quite
useless in Python, if I did not see that the constructor speed can be
faster.
I'm not sure 100% of the result. This is why I'm suggesting to try to
implement the speed optimizations to the constructor of dict first:
1. the optimizations could be not safe. Indeed recently I got a segfault
that I have to investigate
2. maybe dict can be really optimized further. After that, the difference
between dict and frozendict performance could be minimal
That said, maybe there are four good use cases for a frozendict:
1. they could be used for __annotations__ of function objects, and similar
cases
2. they could be used to implement "immutable" modules or classes
3. frozendict can be easily cached, like tuples.
4. as I said, as an alternative to MappingProxyType, since it's very slow
and it's used also in the C code of CPython and in the Python stdlib. This
maybe is not useful because:
a. the speed of MappingProxyType can be improved
b. MappingProxyType can't be replaced because you *really* want and
need a proxy
Date: Fri, 26 Jun 2020 18:47:44 +0200
From: Hans Ginzel <hans(a)matfyz.cz>
To: Hans Ginzel <hans(a)artax.karlin.mff.cuni.cz>
Subject: Access (ordered) dict by index; insert slice
Hello,
thank you for making dict ordered.
Is it planned to access key,value pair(s) by index? See https://stackoverflow.com/a/44687752/2556118 for example. Both for reading and (re)writing?
Is it planned to insert pair(s) on exact index? Or generally to slice? See splice() in Perl, https://perldoc.perl.org/functions/splice.html.
Use case: Represent database table metadata (columns). It is useful as to access columns both by name and by index as to insert column on specific position, https://dev.mysql.com/doc/refman/8.0/en/alter-table.html, “ALTER TABLE ADD COLUMN [FIRST |AFTER col]” (consider default order or table storage size optimisation by aligning).
Thank you in advance,
Hans
PS1: Named tuples cannot be used, are immutable.
PS2: See https://metacpan.org/pod/perlref#Pseudo-hashes:-Using-an-array-as-a-hash