Mailman 3 March 2018 - Pandas-dev

Welcome Pietro to the core team
by Joris Van den Bossche March 29, 2018

March 29, 2018

Hi all, On behalf of the core developers I'd like to welcome Pietro Battiston (@toobaz) to the core dev team. Pietro has been active on a regular basis for the last 2 years, and is especially closely involved in discussions regarding MultiIndexing and did some nice work on that. Thanks for all those contributions, and looking forward to further collaboration and your continued involvement! Joris

4 3

Proposal to change the default number of rows for DataFrame display (lower max_rows)
by Joris Van den Bossche March 28, 2018

March 28, 2018

*[Note for those reading it on the pydata mailing list, please answer to pandas-dev(a)python.org <pandas-dev(a)python.org> to keep discussion centralised there]* Hi all, I am reposting the mail of Clemens below, but with slightly changed focus, as I think the main discussion point is about the number of rows. The proposal in https://github.com/pandas-dev/pandas/pull/17023 is to lower the default number of rows shown when displaying a Series or DataFrame from 60 to 20. Thoughts on that? Best, Joris 2017-11-28 11:57 GMT+01:00 Clemens Brunner <clemens.brunner(a)gmail.com>: > Hello! > > We're currently discussing a change in how data frames are displayed by > default in https://github.com/pandas-dev/pandas/pull/17023. There are two > proposed changes: > > (1) Set pd.options.display.max_columns=0 (previously this was set to 20). > (2) Set pd.options.display.max_rows=20 (previously this was set to 60). > > Change (1) means that the number of printed columns is adapted to fit > within the width of the terminal. If there are too many columns, ellipsis > will be shown to indicate collapsed columns in the middle of the data > frame. This doesn't work if Python is run as a Jupyter kernel (e.g. in a > Jupyter notebook or in IPython QtConsole), in which case the maximum > columns remain 20. > > Example: > ======== > import pandas as pd > import numpy as np > pd.DataFrame(np.random.rand(5, 10)) > > Output before (in a terminal with 100 chars width): > --------------------------------------------------- > 0 1 2 3 4 5 6 \ > 0 0.643979 0.690414 0.018603 0.991478 0.707534 0.376765 0.670848 > 1 0.547836 0.810972 0.054448 0.415112 0.268120 0.904528 0.839258 > 2 0.582256 0.732149 0.284208 0.405197 0.213591 0.715367 0.150106 > 3 0.197348 0.317159 0.051669 0.738405 0.821046 0.179270 0.245793 > 4 0.483466 0.583330 0.999213 0.882883 0.315169 0.045712 0.897048 > > 7 8 9 > 0 0.891467 0.494220 0.713369 > 1 0.601304 0.449880 0.266205 > 2 0.113262 0.360580 0.238833 > 3 0.798063 0.077769 0.471169 > 4 0.262779 0.530565 0.992084 > > Output after: > ------------- > 0 1 2 3 ... 6 7 > 8 9 > 0 0.673621 0.211505 0.943201 0.946548 ... 0.900453 0.612182 > 0.861933 0.710967 > 1 0.670855 0.834449 0.796273 0.785976 ... 0.609954 0.686663 > 0.684582 0.837505 > 2 0.544736 0.814827 0.352893 0.459556 ... 0.650993 0.735943 > 0.279110 0.840203 > 3 0.440125 0.554323 0.745462 0.940896 ... 0.544576 0.224175 > 0.852603 0.509837 > 4 0.225551 0.791834 0.476059 0.321857 ... 0.391165 0.423213 > 0.290683 0.954423 > > [5 rows x 10 columns] > > > Change (2) implies fewer rows are displayed before auto-hiding takes > place. I find that 60 rows almost always causes the terminal to scroll > (most terminals have between 25-40 rows), so reducing the value to 20 > increases the chance that a data frame can be observed on one terminal > page. I'm not including a before/after output since it should be easy to > imagine how this change affects the output. > > Both changes would make Pandas behave similar to R's Tidyverse (which I > really like), but this should not be the main reason why these changes are > a good idea. I mainly like them because these settings make (large) data > frames much nicer to look at. > > Note that these changes affect the default values. Of course, users are > free to change them back in their active Python session. > > Comments to both proposed changes are highly welcome (either here on the > mailing list or at https://github.com/pandas-dev/pandas/pull/17023. > > Clemens > _______________________________________________ > Pandas-dev mailing list > Pandas-dev(a)python.org > https://mail.python.org/mailman/listinfo/pandas-dev >

2 3

Docs translation
by Leticia Portella March 23, 2018

March 23, 2018

Hello! I am wondering if there is any effort being made nowadays to translate the pandas documentation to new languages (in my case, brazilian portuguese). If there is, can you tell me where is this being made so I can help? If not, can you help me start a project on transifex or some other tool to make this possible!? Thank you for yout attention. kind regards, Leticia -- Letícia Portella leportella.com podcast.datascience.pizza

2 1

Chained filtering with lazy evaluation ("where")
by Pietro Battiston March 22, 2018

March 22, 2018

Dear pandas devs, like most (I think) of you, I love how pandas supports chained assignments. And like several other users, I get frustrated when I have to break some chained sequence of calls because a given operation cannot be included. See for instance https://stackoverflow.com/q/11869910/2858145 https://stackoverflow.com/q/40028500/2858145 https://stackoverflow.com/q/44912692/2858145 I ended up noticing that most of the time, the problematic operation is a filtering, since it is typically done as df.loc[condition_on(df)] e.g. df.loc[df['a'] > 3] In R, we would do (something more similar to) df.loc[a>3] ... but we can't in Python syntax. This is not usually a huge deal - one could even claim that "df[df['a'] > 3]" is nicer because it's more explicit. Still, when it's not df but rather a 5 lines chained assignment, one needs to create the df, and then filter it, which is annoying. There are a couple of other solutions: df.filter, adding an ad-hoc method to pandas objects... but I never found any of them general and/or pythonic enough. So I tried with an alternative: lazy evaluation. It took relatively few lines of code, and after some weeks of use, I'm really satisfied of the result: https://github.com/toobaz/generic_utils/blob/master/generic_utils/panda s/where.py (do not bother about the rest of the repo, the file works as a standalone module). This allows to replace df.loc[df['a'] > 2] with df.loc[W['a'] > 2] ... and to apply virtually any operation one would apply to df (more precisely, any operation... which is chainable).¹ As a bonus, one can write a condition and reuse it to filter several pandas objects. I'm writing this email to ask: - whether you have in mind some alternative solution I did not consider to the problem of "unchainable filterings" - whether you have suggestions on how to improve my solution - whether you think this is worth merging in pandas (the amount of monkey patching required is so small that it is not burdensome to keep it separated - it just means one more dependency for users who want to use it) For the records: it currently works only in .loc... and I don't expect this to change: I guess pd.{Series,DataFrame}.__getitem__ already support too many different mechanisms. Supporting .loc as setter should be instead pretty straightforward - it is just lower priority as not used in chaining. Pietro ¹ Only exception (I know of) at the moment: W.loc(axis=1)[.] won't work, because I "taught" it that "loc" is not a callable. Shouldn't be hard to fix.

5 8

Open questions regarding docstrings
by Marc Garcia March 5, 2018

March 5, 2018

Hi there, There are few things regarding the docstrings, that are still open to discussion, in many cases because the numpy convention (or numpy doc examples) is different for the unwritten convention used in most pandas docstrings. Probably you've seen the discussion in GitHub, but I list them here, with the proposed decision (mainly keep the pandas way). If anyone disagrees in any point, please let us know, so we'll change the documentation for the sprint, and do it in the desired way. 1) Starting the docstring just after the opening triple quotes, or in the next line. In pandas it's more common to do it in the next line, so we'll keep it this way. 2) For parameters, showing the default value after the type, or after the description. Numpy does not find it necessary to specify them, and it specified the recommended place is after the description. The proposal (mainly by Joris) is to always have them and after the type, as it's easier to see it. 3) For parameters expecting a string, in the numpy convention examples `str` is used, the proposal is to use `string` instead. 4) For complex types like dicts, I think there is some consensus that is easier to understand the types if using brackets (e.g. "dict of {str: int}" over "dict of str: int"). And same for tuples (e.g. "tuple of (int, str, int)" over "tuple of int, str, int"). For list and sets, the type is simpler (e.g. "list of int" or "set of str"). I propose to use the brackets for list and tuple, and not for list and set, and use `str` over `string` if part of a complex type. 5) For cases where a parameter is optional, so, have a None value by default, meaning the value is not required (as I understand if it was the case of `fillna(value=None)` value wouldn't be optional, as it means is the value used to replace `NaN`). In this case, the proposal is to use as the type, something like "int or float, optional" over "int, float or None (default None)". 6) When the parameter expects something in the form of a Python list, a numpy array, a pandas Series... document it as "array-like" over other options list "iterable" or "numpy.array, Series or list". Thanks!

4 3

March 1st Pandas Development Meeting
by Tom Augspurger March 1, 2018

March 1, 2018

Hi all, We're holding a dev meeting on Thursday March 1st (tomorrow) at 9:00 Eastern / 14:00 UTC. All are welcome to attend. Hangout: https://hangouts.google.com/hangouts/_/2tvkpdso6vh6bhrw2yw7pxtkvae Minutes: https://docs.google.com/document/d/1tGbTiYORHiSPgVMXawiweGJlBw5dOkVJLY-lico… Tom

1 0