Mailman 3 January 2018 - Python-ideas

Chaining coders
by Soni L. 20 Jan '18

20 Jan '18

windows-1252 is based on iso-8859-1. Thus, I'd like to be able to chain coders as follows: bytes.decode("windows-1252-ext", else=lambda r: r.decode("iso-8859-1")) What this "else" does is that it's a lambda, and it gets passed an object with a decode method identical to the bytes decode method, except that it doesn't affect already-decoded characters. In this case, "windows-1252-ext" only includes things in the \x80-\x9F range, leaving it up to "iso-8859-1" to handle the rest. A similar process would happen for encoding: encode with "windows-1252-ext", else = "iso-8859-1". (Technically, "windows-1252-ext" isn't needed - you can use the existing "windows-1252" and combine it with the "iso-8859-1" to get "windows-1252-c1".) This would be a novel way to think of encodings as not just flat translation tables but highly composable translation tables. I have a thing for composition.

2 1

Re: [Python-ideas] Repurpose `assert' into a general-purpose check
by Paul Moore 19 Jan '18

19 Jan '18

Grr, Google Groups gateway messes up reply-to. Apologies to anyone who gets a double-post, please can posters ensure that reply-to is set to the list, and *not* to the Google Groups gateway? Thanks. Paul On 16 January 2018 at 15:54, Paul Moore <p.f.moore(a)gmail.com> wrote: > On 16 January 2018 at 15:37, smarie > <sylvain.marie(a)schneider-electric.com> wrote: > >>> If you, the developer, don't want a check to be disabled, then you >>> shouldn't call it an assertion and use assert. >> >> >> That is exactly what I'm saying. It seems that we both agree that >> applicative value validation is different from asserts, and that assert >> should not be used for applicative value validation. >> For this reason, I do not suggest to go in the direction the OP is >> mentioning but rather to explicitly separate the 2 concepts by creating a >> new statement for value validation. >> >>> >>> The problem with a statement called "validate" is that it will break a >>> huge number of programs that already include functions and methods using >>> that name. >> >> You definitely make a point here. But that would be the case for absolutely >> *any* language evolution as soon as the proposed statements are plain old >> english words. Should it be a show-stopper ? I dont think so. > > Why does this need to be a statement at all? Unlike assert, it's > always executed, so it can be defined as a simple function: > > def validate(test, message): > if not test: > raise ValidartionError(message) > >>> But apart from the use of a keyword, we already have a way to do almost >>> exactly what you want: >>> >>> if not expression: raise ValidationError(message) >>> >>> after defining some appropriate ValidationError class. And it is only a >>> few key presses longer than the proposed: >>> >>> validate expression, ValidationError, message >> >> >> This is precisely what is not good in my opinion: here you do not separate >> <validation means> from <validation intent>. Of course if <validation means> >> is just a "x > 0" statement, it works, but now what if you rely on a >> 3d-party provided validation function (or even yours) such as e.g. >> "is_foo_compliant" ? >> >> if not is_foo_compliant(x): raise ValidationError(message) >> >> What if this third part method raises an exception instead of returning >> False in some cases ? >> >> try: >> if not is_foo_compliant(x): raise ValidationError(message) >> except: >> raise MyValidationError(message) >> >> What if you want to compose this third party function with *another* one >> that returns False but not exceptions ? Say, with an OR ? (one or the other >> should work). Yields: >> >> try: >> if not is_foo_compliant(x): raise ValidationError(message) >> except: >> if not is_bar_compliant(x): >> raise MyValidationError(message) >> >> It starts to be quite ugly, messy... while the applicative intent is clear >> and could be expressed directly as: >> >> validate is_foo_compliant(x) or is_bar_compliant(x) >> ValidationError(message) > > I don't see how a validate statement avoids having to deal with all of > the complexity you mention here. And it's *far* easier to handle this > as a standalone function - if you find a new requirement like the ones > you suggest above, you simply modify the function (and release an > updated version of your package, if you choose to release your code on > PyPI) and you're done. With a new statement, you'd need to raise a > Python feature request, wait for at least the next Python release to > see the modification, and *still* have to support people on older > versions of Python with the unfixed version. > > Also, a validate() function will wor on older versions of Python, all > the way back to Python 2.7 if you want. > > Paul

11 17

I
by Lorena Mesa 19 Jan '18

19 Jan '18

-- __________________________________________________________________ *Lorena Mesa* Co-Organizer, PyLadies Chicago <http://www.meetup.com/Chicago-PyLadies/> Director, Python Software Foundation www.lorenamesa.com @loooorenanicole <https://twitter.com/loooorenanicole> Pronouns: she/her/hers Say what? <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

1 0

Repurpose `assert' into a general-purpose check
by Ivan Pozdeev 17 Jan '18

17 Jan '18

The `assert' statment was created the same as in previous languages like C/C++: a check to only do in debug mode, when you can't yet trust your code to manage and pass around internal data correctly. Examples are array bounds and object state integrity constraints. Unlike C, Python does the aforementioned checks all the time, i.e. it's effectively always in "debug mode". Furthermore, validation checks are an extremily common use case. I have been using assert in my in-house code for input validation for a long time, the only thing preventing me from doing the same in public code is the fact that it only raises AssertionError's while library code is expected to raise TypeError or ValueError on invalid input. The last drop that incited me to do this proposition was https://stackoverflow.com/questions/2142202/what-are-acceptable-use-cases-f… where none of the _seven_ answer authors (beside me) managed to make a _single realistic use case_ for `assert' as a debug-only check. --- So, I'm hereby proposing to: * make `assert' stay in optimized mode * change its syntax to raise other types of exceptions E.g.: "assert condition, type, value" or "assert condition, type, exception_constructor_argument(s)" to maintain backward compatibility. -- Regards, Ivan

17 34

PEP 557 Dataclasses evolution: supporting implicit field creation with __init__ signature introspection
by smarie 16 Jan '18

16 Jan '18

(my first post seems to have been accepted by google groups but rejected by the mailing list. The first repost attempt was completely destructured by outlook > Reposting a 2d time… VERY sorry for the inconvenience, I’m not familiar with google groups) Hello there I recently found out that while I was developing autoclass <https://smarie.github.io/python-autoclass/> in complete, naïve ignorance of the python evolution process, there was a PEP (557, here <https://www.python.org/dev/peps/pep-0557/>) growing, being proposed by Eric V. Smith; and now accepted since december. While this PEP is a GREAT step forward developer-friendly classes in python, I do not feel that it covers all of the use cases I wanted to cover with autoclass. I discussed with Eric about the particular one below and he suggested to post it here: In the case users want to benefit from the code generation while keeping custom __init__ methods the current proposal leads to double declaration as shown in the pep. <https://www.python.org/dev/peps/pep-0557/#custom-init-method> The choice I made with autoclass is opposite/complementary: rather than using class descriptor to define fields, the developer includes all of them in the __init__ method signature, and an optional include/exclude parameter allows him/her to further customize the list if need be. See here <https://smarie.github.io/python-autoclass/#basic>. I think that both ways to declare fields actually make sense, depending on the use case. Compact pure-data classes will be more compact and readable with fields defined as class descriptors. Hybrid data+logic classes with custom constructors will be more compact if the fields used in the custom constructor signature are not defined twice (also in a descriptor). Supporting both ways to declare the fields, possibly both styles in the same class definition (but does it make sense? not sure), would avoid duplication when users want custom constructors. The example in the pep <https://www.python.org/dev/peps/pep-0557/#custom-init-method> would become: @dataclass() class ArgHolder: def __init__(self, *args: Any, **kwargs: Any): pass Does that make sense ? Kind regards Sylvain

1 0

Make functions, methods and descriptor types living in the types module
by Serhiy Storchaka 13 Jan '18

13 Jan '18

Currently the classes of functions (implemented in Python and builtin), methods, and different type of descriptors, generators, etc have the __module__ attribute equal to "builtins" and the name that can't be used for accessing the class. >>> def f(): pass ... >>> type(f) <class 'function'> >>> type(f).__module__ 'builtins' >>> type(f).__name__ 'function' >>> type(f).__qualname__ 'function' >>> import builtins >>> builtins.function Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: module 'builtins' has no attribute 'function' But most of this classes (if not all) are exposed in the types module. I suggest to rename them. Make the __module__ attribute equal to "builtins" and the __name__ and the __qualname__ attributes equal to the name used for accessing the class in the types module. This would allow to pickle references to these types. Currently this isn't possible. >>> pickle.dumps(types.FunctionType) Traceback (most recent call last): File "<stdin>", line 1, in <module> _pickle.PicklingError: Can't pickle <class 'function'>: attribute lookup function on builtins failed And this will help to implement the pickle support of dynamic functions etc. Currently the third-party library that implements this needs to use a special purposed factory function (not compatible with other similar libraries) since types.FunctionType isn't pickleable.

5 4

Re: [Python-ideas] Support WHATWG versions of legacy encodings
by Nathaniel Smith 12 Jan '18

12 Jan '18

On Jan 11, 2018 4:05 AM, "Antoine Pitrou" <solipsis(a)pitrou.net> wrote: Define "widely used". If web-XXX is a superset of windows-XXX, then perhaps web-XXX is "used" in the sense of "used to decode valid windows-XXX data" (but windows-XXX could be used just as well to decode the same data). The question is rather: how often does web-XXX mojibake happen? We're well in the 2010s now and you'd hope that mojibake doesn't happen as often as it used to in, e.g., 1998. I'm not an expert here or anything, but from what we've been hearing it sounds like it must be used by all standard-compliant HTML parsers. I don't *like* the standard much, but I don't think that the stdlib should refuse to handle standard-compliant HTML, or help users handle standard-compliant HTML correctly, just because the HTML standard has unfortunate things in it. We're not going to convince them to change the standard or anything. And this whole thread started with someone said that their mojibake fixing library is having trouble because of this, so clearly mojibake does still exist. Does it help if we reframe it as not that whatwg is "wrong" about windows-1252, but rather that there is this encoding web-1252, and thanks to an interesting quirk of history, in HTML documents the byte sequence b'<meta charset="windows-1252">' indicates a file using this encoding? In fact the mapping between byte sequences and character sets here is so arbitrary that in standards-compliant HTML, the byte sequences b'<meta charset="ascii">', b'<meta charset="iso-8859-1">', and b'<meta charset="latin1">' *also* indicate that the file is encoded using web-1252. (See: https://encoding.spec.whatwg.org/#names-and-labels) -n

3 2

Syntax to import modules before running command from the command line
by Steve Barnes 11 Jan '18

11 Jan '18

Currently invoking `python -c "some;separated;set of commands;"` will, if you need to use any library functions, require one or more import somelib; sections in the execution string. This results in rather complex "one liners". On the other hand `python -m somelib` will load somelib and attempt to execute its `__main__()` or give an error if there isn't one. What I would like to suggest is a mechanism to pre-load libraries before evaluating the -c option as this would allow the use of code from libraries that don't have a `__main__` function, or those that do but it doesn't do what you want. Since -m for module is already taken I would suggest one of: -p for pre-load module -M for load module without attempting to execute `module.__main__()` and without defining "__main__" in the load context or -l for library with the last two having the advantage of appearing next to -m in the --help output. This would change, (for a trivial example): `python -c"import numpy;print(numpy.pi);"` to: `python -M numpy -c"print(numpy.pi);"` -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.

6 9

Re: [Python-ideas] Allow to compile debug extension against releasePython in Windows
by Ivan Pozdeev 10 Jan '18

10 Jan '18

On 08.01.2018 0:11, Steve Dower wrote: > > It’s not a good idea. You end up with two different C runtimes in > memory that cannot communicate, and many things will not work properly. > > If you compile your debug build extension with the non-debug CRT (/MD > rather than /MDd) you will lose asserts, but otherwise it will work > fine and the quoted code picks the release lib. > Distutils' designers seem to have thought differently. Whether the extension is linked against pythonxx_d.lib is governed by the --debug switch to the `build' command rather than the type of the running Python. Compiler optimization flags are inserted according to it, too. As a consequence, * I cannot install an extension into debug Python at all 'cuz `bdist_*' and `install' commands don't support --debug and invoke `debug' internally without it. * Neither can I compile an extension for release Python without optimizations. I'm at a loss here. Either I'm missing something, or with the current build system, it's impossible to debug extensions! > > Or if you like, when you install Python 3.5 or later there are > advanced options to install debug symbols and binaries. You can use a > proper debug build against the debug binaries (python_d.exe). > > Cheers, > > Steve > > Top-posted from my Windows phone > > *From: *Ivan Pozdeev via Python-ideas <mailto:python-ideas@python.org> > *Sent: *Saturday, December 30, 2017 13:01 > *To: *python-ideas(a)python.org <mailto:python-ideas@python.org> > *Subject: *[Python-ideas] Allow to compile debug extension against > releasePython in Windows > > The Windows version of pyconfig.h has the following construct: > > if defined(_DEBUG) > > pragma comment(lib,"python37_d.lib") > > elif defined(Py_LIMITED_API) > > pragma comment(lib,"python3.lib") > > else > > pragma comment(lib,"python37.lib") > > endif /* _DEBUG */ > > which fails the compilation of a debug version of an extension. Making > > debugging it... difficult. > > Perhaps we could define some other constant? > > I'm not sure whether such compilation is a good idea in general, so > > asking here at first. > > -- > > Regards, > > Ivan > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas(a)python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Regards, Ivan

3 5

pdb to support running modules
by Mario Corchero 10 Jan '18

10 Jan '18

Hello All, Since PEP 338 <http://www.python.org/dev/peps/pep-0338/> we can run python modules as a script via `python -m module_name` but there is no way to run pdb on those (AFAIK). The proposal is to add a new argument "-m" to the pdb module to allow users to run `python -m pdb -m my_module_name` This is especially useful when working on cli tools that use entrypoints in setup.py, as there is no other way to run them. I have a possible implementation here https://github.com/python/cpython/pull/4752 I quite often use pdb with scripts, being able to run modules would be useful as well. What do you think? Regards! Mario

6 5