Mailman 3 March 2017 - Python-ideas

Re: [Python-ideas] What about regexp string litterals : re".*" ?
by Ryan Gonzalez March 30, 2017

March 30, 2017

I feel like that borders on a bit too wordy... Personally, I'd like to see something like Felix's regular definitions: http://felix-lang.org/share/src/web/tut/regexp_01.fdoc#Regular_definitions.… -- Ryan (ライアン) Yoko Shimomura > ryo (supercell/EGOIST) > Hiroyuki Sawano >> everyone else http://refi64.com On Mar 29, 2017 3:30 PM, "Abe Dillon" <abedillon(a)gmail.com> wrote: My 2 cents is that regular expressions are pretty un-pythonic because of their horrible readability. … [View More]

3 2

Add pathlib.Path.write_json and pathlib.Path.read_json
by Ram Rachum March 29, 2017

March 29, 2017

Hi guys, What do you think about adding methods pathlib.Path.write_json and pathlib.Path.read_json , similar to write_text, write_bytes, read_text, read_bytes? This would make writing / reading JSON to a file a one liner instead of a two-line with clause. Thanks, Ram.

18 30

Proposal: Query language extension to Python (PythonQL)
by Pavel Velikhov March 28, 2017

March 28, 2017

Hi folks! We started a project to extend Python with a full-blown query language about a year ago. The project is call PythonQL, the links are given below in the references section. We have implemented what is kind of an alpha version now, and gained some experience and insights about why and where this is really useful. So I’d like to share those with you and gather some opinions whether you think we should try to include these extensions in the Python core. Intro What we have done is (… [View More]mostly) extended Python’s comprehensions with group by, order by, let and window clauses, which can come in any order, thus comprehensions become a query language a bit cleaner and more powerful than SQL. And we added a couple small convenience extensions, like a We have identified three top motivations for folks to use these extensions: Our Motivations 1. This can become a standard for running queries against database systems. Instead of learning a large number of different SQL dialects (the pain point here are libraries of functions and operators that are different for each vendor), the Python developer needs only to learn PythonQL and he can query any SQL and NoSQL database. 2. A single PythonQL expression can integrate a number of databases/files/memory structures seamlessly, with the PythonQL optimizer figuring out which pieces of plans to ship to which databases. This is a cool virtual database integration story that can be very convenient, especially now, when a lot of data scientists use Python to wrangle the data all day long. 3. Querying data structures inside Python with the full power of SQL (and a bit more) is also really convenient on its own. Usually folks that are well-versed in SQL have to resort to completely different means when they need to run a query in Python on top of some data structures. Current Status We have PythonQL running, its installed via pip and an encoding hack, that runs our preprocessor. We currently compile PythonQL into Python using our executor functions and execute Python subexpressions via eval. We don’t do any optimization / rewriting of queries into languages of underlying systems. And the query processor is basic too, with naive implementations of operators. But we’ve build DBMS systems before, so if there is a good amount of support for this project, we’ll be able to build a real system here. Your take on this Extending Python’s grammar is surely a painful thing for the community. We’re now convinced that it is well worth it, because of all the wonderful functionality and convenience this extension offers. We’d like to get your feedback on this and maybe you’ll suggest some next steps for us. References PythonQL GitHub page: https://github.com/pythonql/pythonql <https://github.com/pythonql/pythonql> PythonQL Intro and Tutorial (this is all User Documentation we have right now): https://github.com/pythonql/pythonql/wiki/PythonQL-Intro-and-Tutorial <https://github.com/pythonql/pythonql/wiki/PythonQL-Intro-and-Tutorial> A use-case of querying Event Logs and doing Process Mining with PythonQL: https://github.com/pythonql/pythonql/wiki/Event-Log-Querying-and-Process-Mi… <https://github.com/pythonql/pythonql/wiki/Event-Log-Querying-and-Process-Mi…> PythonQL demo site: www.pythonql.org <http://www.pythonql.org/> Best regards, PythonQL Team [View Less]

11 24

Adding an 'errors' argument to print
by Ryan Gonzalez March 28, 2017

March 28, 2017

Recently, I was working on a Windows GUI application that ends up running ffmpeg, and I wanted to see the command that was being run. However, the file name had a Unicode character in it (it's a Sawano song), and when I tried to print it to the console, it crashed during the encode/decode. (The encoding used in cmd doesn't support Unicode characters.) The workaround was to do: print(mystring.encode(sys.stdout.encoding, errors='replace).decode(sys.stdout.encoding)) Not fun, especially since … [View More]

8 8

Re: [Python-ideas] Adding an 'errors' argument to print
by Victor Stinner March 27, 2017

March 27, 2017

print(msg) calls sys.stdout.write(msg): write() expects text, not bytes. I dislike the idea of putting encoding options in print. It's too specific. What if tomorrow you replace print() with file.write()? Do you want to add errors there too? No, it's better to write own formatter function as shown in a previous email. Victor Le 25 mars 2017 8:50 PM, "Michel Desmoulin" <desmoulinmichel(a)gmail.com> a écrit : Le 24/03/2017 à 17:37, Victor Stinner a écrit : > *If* we change … [View More]

6 7

Re: [Python-ideas] Proposal: Query language extension to Python (PythonQL)
by Gerald Britton March 27, 2017

March 27, 2017

> > On 25 March 2017 at 11:24, Pavel Velikhov <pavel.velikhov at gmail.com> > wrote: > > No, the current solution is temporary because we just don’t have the > > manpower to > > implement the full thing: a real system that will rewrite parts of > PythonQL > > queries and > > ship them to underlying databases. We need a real query optimizer and > smart > > wrappers > > for this purpose. But we’ll build one of these for demo purposes … [View More]soon > > (either a Spark > > wrapper or a PostgreSQL wrapper). > One thought, if you're lacking in manpower now, then proposing > inclusion into core Python means that the core dev team will be taking > on an additional chunk of code that is already under-resourced. That > rings alarm bells for me - how would you imagine the work needed to > merge PythonQL into the core Python grammar would be resourced? > I should say that in practice, I think that the solution is relatively > niche, and overlaps quite significantly with existing Python features, > so I don't really see a compelling case for inclusion. The parallel > with C# and LINQ is interesting here - LINQ is a pretty cool > technology, but I don't see it in widespread use in general-purpose C# > projects (disclaimer: I don't get to see much C# code, so my experience is limited). I see lots of C# code, but (thankfully) not so much LINQ to SQL. Yes, it is a cool technology. But I sometimes have a problem with the SQL it generates. Since I'm also a SQL developer, I'm sensitive to how queries are constructed, for performance reasons, as well as how they look, for readability and aesthetic reasons. LINQ queries can generate poorly-performing SQL, since LINQ is a basically a translator, but not an AI. As far as appearances go, LINQ queries can look pretty gnarly, especially if they include sub queries or a few joins. That makes it hard for the SQL dev (me!) to read and understand if there are performance problems (which there often are, in my experience) So, I would tend to code the SQL separately and put it in a SQL view, function or stored procedure. I can still parse the results with LINQ (not LINQ to SQL), which is fine. For similar reasons, I'm not a huge fan of ORMs either. Probably my bias towards designing the database first and building up queries to meet the business goals before writing a line of Python, C#, or the language de jour. -- Gerald Britton, MCSE-DP, MVP LinkedIn Profile: http://ca.linkedin.com/in/geraldbritton [View Less]

4 7

(no subject)
by Gerald Britton March 27, 2017

March 27, 2017

>* On 25 Mar 2017, at 15:51, Gerald Britton <gerald.britton at gmail.com <https://mail.python.org/mailman/listinfo/python-ideas>> wrote: > *> >* On 25 March 2017 at 11:24, Pavel Velikhov <pavel.velikhov at gmail.com <http://gmail.com> <http://gmail.com/ <http://gmail.com/>>> wrote: > *>* > No, the current solution is temporary because we just don’t have the > *>* > manpower to > *>* > implement the full thing: a real … [View More]system that will rewrite parts of PythonQL > *>* > queries and > *>* > ship them to underlying databases. We need a real query optimizer and smart > *>* > wrappers > *>* > for this purpose. But we’ll build one of these for demo purposes soon > *>* > (either a Spark > *>* > wrapper or a PostgreSQL wrapper). > *>* One thought, if you're lacking in manpower now, then proposing > *>* inclusion into core Python means that the core dev team will be taking > *>* on an additional chunk of code that is already under-resourced. That > *>* rings alarm bells for me - how would you imagine the work needed to > *>* merge PythonQL into the core Python grammar would be resourced? > *>* I should say that in practice, I think that the solution is relatively > *>* niche, and overlaps quite significantly with existing Python features, > *>* so I don't really see a compelling case for inclusion. The parallel > *>* with C# and LINQ is interesting here - LINQ is a pretty cool > *>* technology, but I don't see it in widespread use in general-purpose C# > *>* projects (disclaimer: I don't get to see much C# code, so my > *>* experience is limited). > *> >* I see lots of C# code, but (thankfully) not so much LINQ to SQL. Yes, it is a cool technology. But I sometimes have a problem with the SQL it generates. Since I'm also a SQL developer, I'm sensitive to how queries are constructed, for performance reasons, as well as how they look, for readability and aesthetic reasons. > *> >* LINQ queries can generate poorly-performing SQL, since LINQ is a basically a translator, but not an AI. As far as appearances go, LINQ queries can look pretty gnarly, especially if they include sub queries or a few joins. That makes it hard for the SQL dev (me!) to read and understand if there are performance problems (which there often are, in my experience) > *> We want to go beyond being a basic translator. Especially if the common use-case will be integrating multiple databases. We can also introduce decent-looking hints (maybe not always decent looking) to generate better plans. Not sure about asethetics though... >* So, I would tend to code the SQL separately and put it in a SQL view, function or stored procedure. I can still parse the results with LINQ (not LINQ to SQL), which is fine. > *> >* For similar reasons, I'm not a huge fan of ORMs either. Probably my bias towards designing the database first and building up queries to meet the business goals before writing a line of Python, C#, or the language de jour. > * This sounds completely reasonable, but this means you’re tied to a specific DBMS (especially if you’re using a lot of built-in functions that are usually very specific to a database). PythonQL (when it has enough functionality) should give you independence. True though not always needed. e.g. at present I'm working for a large company with thousands of db servers in all the popular flavors. The probability of changing even one of them to a different vendor is essentially zero. The costs and dependencies far outweigh any hoped-for advantage. At the same time, I'm happy to optimize the SQL for different target environments. If I lack the specific expertise, I know where to go to find it. The Adapter pattern helps here. It's actually more important for me to build queries that can be used in multiple client languages. We're using Java, C++, C#, F#, VB, ... and Python, of course (and probably others that I don't know we use). I can optimize the query once and not worry about the clients messing it up. -- Gerald Britton, MCSE-DP, MVP LinkedIn Profile: http://ca.linkedin.com/in/geraldbritton [View Less]

1 0

IDEA
by Faaiz Asim March 27, 2017

March 27, 2017

Hi, I am currently enrolled in computer science program and am currently in 6th semester. I am comfortable using c++,java,python in general. I know this is a little late for proposing an idea but i was busy in exams. Besides i wanted to get into something where i was determined to contribute to the society. It took a little while but i found this ultimately. I think i have a good idea for a project that might further facilitate the use of python. My experience with python suggests that python … [View More]

4 3

PEP: Python Documentation Translations
by Julien Palard March 23, 2017

March 23, 2017

Hi, Here is the follow-up to the "Translated Python documentation" thread on python-dev [1]_, as a PEP describing the steps to make Python Documentation Translation official on accessible on docs.python.org. .. [1] [Python-Dev] Translated Python documentation (https://mail.python.org/pipermail/python-dev/2017-February/147416.html) You can also read it online at https://github.com/JulienPalard/peps/blob/master/pep-0xxx.rst ============================== PEP: XXX Title: Python Documentation … [View More]

7 11

Third party module in standard library
by George Fischhof March 23, 2017

March 23, 2017

Hi Guys, I would like to ask You: What is the process to propose a module to be part of the standard library? I would like to propose the following modules: requests https://pypi.python.org/pypi/requests and xmltodict https://pypi.python.org/pypi/xmltodict Both of them makes the life easier, and more simple ;-) Of course there are several similar libraries -- I know -- and actually several good modules could be part of standard library. ... Kind regards, George

3 2