Mailman 3 February 2016 - Pandas-dev

Re: [Pandas-dev] Thoughts on adopting a 1-PR-1-commit policy?
by Wes McKinney June 8, 2016

June 8, 2016

hey Jan, I'm adding the mailing list back. Several comments inline On Sun, Jan 17, 2016 at 1:09 AM, Jan Schulz <jasc(a)gmx.net> wrote: > Hi, > > Just a different opinion: I like having commits do one logical thing > and not squash multiple "logical complete"things together (this means > that commit is a logical step, not the PR and that the commits should > be clean and not contain "fixup", "typo" style commits). During the > categorical work, I found that a few times I regretted that I couldn't > go back to look up the specific change in a commit and look up what > and why that commit was done because it was all mixed up with the rest > of the squashed commits in that PR. > I'm not suggesting that you should have to squash your commits inside the PR. This only concerns how *patches are applied to pandas's master branch". Ideally, no squashing occurs inside the developer branch (so the "story", so to speak, about the patch is preserved), but what the Apache patch tool does is - Turns a multi-commit PR into a single-commit patch - Puts the individual commit hashes in the commit message; so you can always visit the original commits - Puts the description from the PR into the commit message - Cherry-picks instead of merging, so you can observe evolution of pandas/master in a clear and linear way > My feeling was always that squashes are performed because the rebases > are so hard because of multiple PRs fixing stuff in the same files at > the same time. If this refactorings come through (both better > separation of backend-frontend specific code and the suggested split > up of frame.py, etc), I think this is not so much a problem anymore. > > I'm not sure where the commit history with merges is a problem: during > balme (in github, never used git itself), I don't see any merges?! > Here is something that would be very hard: create release notes given the commit history. > So my suggestion would go in a different direction: better commits in > the PRs with proper commit messages (not only the headline but also > explanations in the message. > > Jan > -- > Jan Schulz > mail: jasc(a)gmx.net > web: http://www.katzien.de > > > On 17 January 2016 at 07:29, Wes McKinney <wesmckinn(a)gmail.com> wrote: >> The script performs the squashing automatically, so users can squash >> manually if they wish or let us do it. >> >> On Sat, Jan 16, 2016 at 8:11 PM, Jeff Reback <jeffreback(a)gmail.com> wrote: >>> seems find to use the cherry picking script >>> >>> though I don't think should relax users from squashing >>> >>>> On Jan 16, 2016, at 6:46 PM, Wes McKinney <wesmckinn(a)gmail.com> wrote: >>>> >>>> Copying the mailing list. Indeed makes rebasing unnecessary if there >>>> are no cherry-pick conflicts. >>>> >>>> On Sat, Jan 16, 2016 at 3:34 PM, Tom Augspurger >>>> <tom.augspurger88(a)gmail.com> wrote: >>>>> Rebasing can be tough for new contributors, so for that alone I'd say let's try it. >>>>> >>>>> -Tom >>>>> >>>>>> On Jan 16, 2016, at 5:20 PM, Wes McKinney <wesmckinn(a)gmail.com> wrote: >>>>>> >>>>>> I've grown very fond of the PR cherry-picking style used in many >>>>>> Apache projects. >>>>>> >>>>>> Here's an example of a very large commit to Apache Spark that was >>>>>> performed in this fashion: >>>>>> >>>>>> https://github.com/apache/spark/commit/2fe0a1aaeebbf7f60bd4130847d738c29f1e… >>>>>> >>>>>> If you compare pandas's commit history with a project like this, >>>>>> you'll see it is much easier to follow because there is one commit for >>>>>> each patch to the project, rather than a merge commit plus 1 or more >>>>>> merged commits (depending on whether the person merging the PR did an >>>>>> interactive rebase). >>>>>> >>>>>> The script to do this is not too complex, and is even less complex for >>>>>> pandas because we do not use JIRA: >>>>>> >>>>>> https://github.com/apache/spark/blob/master/dev/merge_spark_pr.py >>>>>> >>>>>> I've been using a pared down version of the script in Ibis: >>>>>> >>>>>> https://github.com/cloudera/ibis/blob/master/dev/merge-pr.py >>>>>> >>>>>> Here is an example of what a merge commit with multiple subcommits >>>>>> looks like using this tool: >>>>>> >>>>>> https://github.com/cloudera/ibis/commit/eafabe060dcaaea0a6076342eaa374929b9… >>>>>> >>>>>> It's pretty easy to use: run the script and enter the PR # you are >>>>>> merging. It automatically squashes and closes the merged PR. >>>>>> >>>>>> Let me know if this is something that would interest the team. I know >>>>>> there are varying opinions on the GitHub Green Button =) >>>>>> >>>>>> - Wes >>>>>> _______________________________________________ >>>>>> Pandas-dev mailing list >>>>>> Pandas-dev(a)python.org >>>>>> https://mail.python.org/mailman/listinfo/pandas-dev >>>> _______________________________________________ >>>> Pandas-dev mailing list >>>> Pandas-dev(a)python.org >>>> https://mail.python.org/mailman/listinfo/pandas-dev >> _______________________________________________ >> Pandas-dev mailing list >> Pandas-dev(a)python.org >> https://mail.python.org/mailman/listinfo/pandas-dev

5 12

Re: [Pandas-dev] Rewriting some of internals of pandas in C/C++? / Roadmap
by Jeff Reback March 24, 2016

March 24, 2016

Here are some of my thoughts about pandas Roadmap / status and some responses to Wes's thoughts. In the last few (and upcoming) major releases we have been made the following changes: - dtype enhancements (Categorical, Timedelta, Datetime w/tz) & making these first class objects - code refactoring to remove subclassing of ndarrays for Series & Index - carving out / deprecating non-core parts of pandas - datareader - SparsePanel, WidePanel & other aliases (TImeSeries) - rpy, rplot, irow et al. - google-analytics - API changes to make things more consistent - pd.rolling/expanding * -> .rolling/expanding (this is in master now) - .resample becoming a full defered like groupby. - multi-index slicing along any level (obviates need for .xs) and allows assignment - .loc/.iloc - for the most part obviates use of .ix - .pipe & .assign - plotting accessors - fixing of the sorting API - many performance enhancements both micro & macro (e.g. release GIL) Some on-deck enhancements are (meaning these are basically ready to go in): - IntervalIndex (and eventually make PeriodIndex just a sub-class of this) - RangeIndex so lots of changes, though nothing really earth shaking, just more convenience, reducing magicness somewhat and providing flexibility. Of course we are getting increasing issues, mostly bug reports (and lots of dupes), some edge case enhancements which can add to the existing API's and of course, requests to expand the (already) large code to other usecases. Balancing this are a good many pull-requests from many different users, some even deep into the internals. Here are some things that I have talked about and could be considered for the roadmap. Disclaimer: I do work for Continuum but these views are of course my own; furthermore obviously I am a bit more familiar with some of the 'sponsored' open-source libraries, but always open to new things. - integration / automatic deferral to numba for JIT (this would be thru .apply) - automatic deferal to dask from groubpy where appropriate / maybe a .to_parallel (to simply return a dask.DataFrame object) - incorporation of quantities / units (as part of the dtype) - use of DyND to allow missing values for int dtypes - make Period a first class dtype. - provide some copy-on-write semantics to alleviate the chained-indexing issues which occasionaly come up with the mis-use of the indexing API - allow a 'policy' to automatically provide column blocks for dict-like input (e.g. each column would be a block), this would allow a pass-thru API where you could put in numpy arrays where you have views and have them preserved rather than copied automatically. Note that this would also allow what I call 'split' where a passed in multi-dim numpy array could be split up to individual blocks (which actually gives a nice perf boost after the splitting costs). In working towards some of these goals. I have come to the opinion that it would make sense to have a neutral API protocol layer that would allow us to swap out different engines as needed, for particular dtypes, or *maybe* out-of-core type computations. E.g. imagine that we replaced the in-memory block structure with a bclolz / memap type; in theory this should be 'easy' and just work. I could also see us adopting *some* of the SFrame code to allow easier interop with this API layer. In practice, I think a nice API layer would need to be created to make this clean / nice. So this comes around to Wes's point about creating a c++ library for the internals (and possibly even some of the indexing routines). In an ideal world, or course this would be desirable. Getting there is a bit non-trivial I think, and IMHO might not be worth the effort. I don't really see big performance bottlenecks. We *already* defer much of the computation to libraries like numexpr & bottleneck (where appropriate). Adding numba / dask to the list would be helpful. I think that almost all performance issues are the result of: a) gross misuse of the pandas API. How much code have you seen that does df.apply(lambda x: x.sum()) b) routines which operate column-by-column rather block-by-block and are in python space (e.g. we have an issue right now about .quantile) So I am glossing over a big goal of having a c++ library that represents the pandas internals. This would by definition have a c-API that so you *could* use pandas like semantics in c/c++ and just have it work (and then pandas would be a thin wrapper around this library). I am not averse to this, but I think would be quite a big effort, and not a huge perf boost IMHO. Further there are a number of API issues w.r.t. indexing which need to be clarified / worked out (e.g. should we simply deprecate []) that are much easier to test / figure out in python space. I also thing that we have quite a large number of contributors. Moving to c++ might make the internals a bit more impenetrable that the current internals. (though this would allow c++ people to contribute, so that might balance out). We have a limited core of devs whom right now are familar with things. If someone happened to have a starting base for a c++ library, then I might change opinions here. my 4c. Jeff On Thu, Dec 24, 2015 at 7:18 PM, Wes McKinney <wesmckinn(a)gmail.com> wrote: > Deep thoughts during the holidays. > > I might be out of line here, but the interpreter-heaviness of the > inside of pandas objects is likely to be a long-term liability and > source of performance problems and technical debt. > > Has anyone put any thought into planning and beginning to execute on a > rewrite that moves as much as possible of the internals into native / > compiled code? I'm talking about: > > - pandas/core/internals > - indexing and assignment > - much of pandas/core/common > - categorical and custom dtypes > - all indexing mechanisms > > I'm concerned we've already exposed too much internals to users, so > this might lead to a lot of API breakage, but it might be for the > Greater Good. As a first step, beginning a partial migration of > internals into some C++ classes that encapsulate the insides of > DataFrame objects and implement indexing and block-level manipulations > would be a good place to start. I think you could do this wouldn't too > much disruption. > > As part of this internal retooling we might give consideration to > alternative data structures for representing data internal to pandas > objects. Now in 2015/2016, continuing to be hamstrung by NumPy's > limitations feels somewhat anachronistic. User code is riddled with > workarounds for data type fidelity issues and the like. Like, really, > why not add a bitndarray (similar to ilanschnell/bitarray) for storing > nullness for problematic types and hide this from the user? =) > > Since we are now a NumFOCUS-sponsored project, I feel like we might > consider establishing some formal governance over pandas and > publishing meetings notes and roadmap documents describing plans for > the project and meetings notes from committers. There's no real > "committer culture" for NumFOCUS projects like there is with the > Apache Software Foundation, but we might try leading by example! > > Also, I believe pandas as a project has reached a level of importance > where we ought to consider planning and execution on larger scale > undertakings such as this for safeguarding the future. > > As for myself, well, I have my hands full in Big Data-land. I wish I > could be helping more with pandas, but there a quite a few fundamental > issues (like data interoperability nested data handling and file > format support — e.g. Parquet, see > > http://wesmckinney.com/blog/the-problem-with-the-data-science-language-wars/ > ) > preventing Python from being more useful in industry analytics > applications. > > Aside: one of the bigger mistakes I made with pandas's API design was > making it acceptable to call class constructors — like > pandas.DataFrame — directly (versus factory functions). Sorry about > that! If we could convince everyone to start writing pandas.data_frame > or dataframe instead of using the class reference it would help a lot > with code cleanup. It's hard to plan for these things — NumPy > interoperability seemed a lot more important in 2008 than it does now, > so I forgive myself. > > cheers and best wishes for 2016, > Wes > _______________________________________________ > Pandas-dev mailing list > Pandas-dev(a)python.org > https://mail.python.org/mailman/listinfo/pandas-dev >

5 41

On bug-fix releases and maintenance branches
by Joris Van den Bossche March 16, 2016

March 16, 2016

Hi all, I wanted to stir some discussion on pandas its policy on bug-fx releases and upgrading pains. First some context: *Context part 1*: Currently we do not use maintenance branches for bugfix releases, and we actually also do not *really *do bugfix releases. We just develop further on master, and try to not merge breaking changes the first weeks/months, so we can do a minor kind of bug-fix release (but usually also with a lot of new features). But we don't, for example, backport fixes of regressions if they are fixed after master is pointing to the next major release. *Context part 2*: pandas is not yet that stable, in the sense that there are still quite some breaking changes in each release. I am not arguing for not doing these breaking changes, as some of these changes are really needed to clean up the API (although there are also arguments for that, but I think that is another discussion). This has the consequence that updating your pandas version is not always that pleasant. Sidenote: I have not that much experience with using pandas in a larger company or in larger codebases that need to be upgraded, rather with just my own code for my PhD. So it is difficult for me to judge on how much this is a problem or if bug-fx releases would help. Questions: - What are other people's experiences with upgrading pandas? And would more bug-fix releases actually ease the upgrading? - Do we want to do more bug-fix releases? - Having a maintenance branch and backporting fixes is extra work. Would we be able to handle this? Would it be worth the effort? (It has been mentioned before, but I think the main point raised was lack of manpower to maintain separate branches) To put it another way. In our whatsnew notice there is "*We recommend that all users upgrade to this version*", but I am actually not sure we should recommend that. I personally do not always recommend that no matter what *without careful consideration*. Regards, Joris

4 6

Re: [Pandas-dev] ANN: pandas v0.18.0rc1 - RELEASE CANDIDATE
by Jeff Feb. 28, 2016

Feb. 28, 2016

So you are probably reading sas7bdat, which was put in AFTER 0.18.0rc1 was cut (if you are reading xport format then you are good to go), otherwise you may want to wait a bit for 0.18.0rc2. On Sunday, February 28, 2016 at 1:42:53 PM UTC-5, John E wrote: > > OK, thanks, I got it. Although... I would consider pandas.pydata.org to > be a common end user gateway and if one starts there they will read "We > recommend that *all *users upgrade to this version." And then if they > scroll down a short distance they will see a single line instruction for > installing via conda: "conda install pandas=v0.18.0rc1 -c pandas". > > And also somewhat confusing to me about pandas.pydata.org is that looking > to the right, you have a choice of RC, dev, and previous releases, but > nothing that says something like "current, stable release". > > Anyways.... quite possibly this is confusing only to me, and not others, > but I thought I'd mention it just in case. FWIW. > > I've now installed 0.18.0rc1 and will try to test out some of the newer > features. I'm really interested to see how well the SAS reader works (i.e. > how fast). I hate SAS myself, but this would be a really, really nice > feature for my organization and likely increase adoption of python & pandas. > > > > On Sunday, February 28, 2016 at 12:03:45 PM UTC-5, Jeff wrote: >> >> >> These are pre-releases. In other words, we would want the community to >> test out before an official release, and see if there are any show >> stoppers. The docs are setup for the official releases. These are not put >> into official channels at all (that is the point), e.g. not on PyPi, nor in >> the conda main channels. Only official releases will go there. >> >> Generally we will try to do release candidates before major changes, but >> not before minor changes. >> >> So the official release of 0.18.0 has not happened yet! (in fact going to >> do a v0.18.0rc2 next week). >> >> We would love for you to test out! >> >> Jeff >> >> >> >> >> On Sunday, February 28, 2016 at 11:50:57 AM UTC-5, John E wrote: >>> >>> I hope this doesn't come across as a trivial, semantical question, but... >>> >>> The initial releases of the last 2 or so versions have been labelled as >>> "release candidates" but still say "We recommend that all >>> users upgrade to this version." >>> >>> So this is a little confusing to me for using pandas in a production >>> environment. "Release candidate" seems to suggest that you should wait for >>> 0.18.1, but the note unambiguously says not to wait. So which >>> interpretation is recommended for a production environment? >>> >>> >>> On Saturday, February 13, 2016 at 7:53:18 PM UTC-5, Jeff wrote: >>>> >>>> Hi, >>>> >>>> I'm pleased to announce the availability of the first release candidate >>>> of Pandas 0.18.0. >>>> Please try this RC and report any issues here: Pandas Issues >>>> <https://github.com/pydata/pandas/issues> >>>> We will be releasing officially in 1-2 weeks or so. >>>> >>>> **RELEASE CANDIDATE 1** >>>> >>>> This is a major release from 0.17.1 and includes a small number of API >>>> changes, several new features, >>>> enhancements, and performance improvements along with a large number of >>>> bug fixes. We recommend that all >>>> users upgrade to this version. >>>> >>>> Highlights include: >>>> >>>> - pandas >= 0.18.0 will no longer support compatibility with Python >>>> version 2.6 GH7718 <https://github.com/pydata/pandas/issues/7718> or >>>> version 3.3 GH11273 <https://github.com/pydata/pandas/issues/11273> >>>> - Moving and expanding window functions are now methods on Series >>>> and DataFrame similar to .groupby like objects, see here >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >>>> . >>>> - Adding support for a RangeIndex as a specialized form of the >>>> Int64Index for memory savings, see here >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >>>> . >>>> - API breaking .resample changes to make it more .groupby like, see >>>> here >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >>>> - Removal of support for positional indexing with floats, which was >>>> deprecated since 0.14.0. This will now raise a TypeError, see here >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >>>> - The .to_xarray() function has been added for compatibility with >>>> the xarray package <http://xarray.pydata.org/en/stable/> see here >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >>>> . >>>> - Addition of the .str.extractall() method >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…>, >>>> and API changes to the the .str.extract() method >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…>, >>>> and the .str.cat() method >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >>>> - pd.test() top-level nose test runner is available GH4327 >>>> <https://github.com/pydata/pandas/issues/4327> >>>> >>>> See the Whatsnew >>>> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html> for >>>> much more information. >>>> >>>> Best way to get this is to install via conda >>>> <http://pandas-docs.github.io/pandas-docs-travis/install.html#installing-pan…> from >>>> our development channel. Builds for osx-64,linux-64,win-64 for Python 2.7 >>>> and Python 3.5 are all available. >>>> >>>> conda install pandas=v0.18.0rc1 -c pandas >>>> >>>> Thanks to all who made this release happen. It is a very large release! >>>> >>>> Jeff >>>> >>>>

1 0

Re: [Pandas-dev] ANN: pandas v0.18.0rc1 - RELEASE CANDIDATE
by Jeff Feb. 28, 2016

Feb. 28, 2016

These are pre-releases. In other words, we would want the community to test out before an official release, and see if there are any show stoppers. The docs are setup for the official releases. These are not put into official channels at all (that is the point), e.g. not on PyPi, nor in the conda main channels. Only official releases will go there. Generally we will try to do release candidates before major changes, but not before minor changes. So the official release of 0.18.0 has not happened yet! (in fact going to do a v0.18.0rc2 next week). We would love for you to test out! Jeff On Sunday, February 28, 2016 at 11:50:57 AM UTC-5, John E wrote: > > I hope this doesn't come across as a trivial, semantical question, but... > > The initial releases of the last 2 or so versions have been labelled as > "release candidates" but still say "We recommend that all > users upgrade to this version." > > So this is a little confusing to me for using pandas in a production > environment. "Release candidate" seems to suggest that you should wait for > 0.18.1, but the note unambiguously says not to wait. So which > interpretation is recommended for a production environment? > > > On Saturday, February 13, 2016 at 7:53:18 PM UTC-5, Jeff wrote: >> >> Hi, >> >> I'm pleased to announce the availability of the first release candidate >> of Pandas 0.18.0. >> Please try this RC and report any issues here: Pandas Issues >> <https://github.com/pydata/pandas/issues> >> We will be releasing officially in 1-2 weeks or so. >> >> **RELEASE CANDIDATE 1** >> >> This is a major release from 0.17.1 and includes a small number of API >> changes, several new features, >> enhancements, and performance improvements along with a large number of >> bug fixes. We recommend that all >> users upgrade to this version. >> >> Highlights include: >> >> - pandas >= 0.18.0 will no longer support compatibility with Python >> version 2.6 GH7718 <https://github.com/pydata/pandas/issues/7718> or >> version 3.3 GH11273 <https://github.com/pydata/pandas/issues/11273> >> - Moving and expanding window functions are now methods on Series and >> DataFrame similar to .groupby like objects, see here >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >> . >> - Adding support for a RangeIndex as a specialized form of the >> Int64Index for memory savings, see here >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >> . >> - API breaking .resample changes to make it more .groupby like, see >> here >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >> - Removal of support for positional indexing with floats, which was >> deprecated since 0.14.0. This will now raise a TypeError, see here >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >> - The .to_xarray() function has been added for compatibility with the xarray >> package <http://xarray.pydata.org/en/stable/> see here >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >> . >> - Addition of the .str.extractall() method >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…>, >> and API changes to the the .str.extract() method >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…>, >> and the .str.cat() method >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> >> - pd.test() top-level nose test runner is available GH4327 >> <https://github.com/pydata/pandas/issues/4327> >> >> See the Whatsnew >> <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html> for much >> more information. >> >> Best way to get this is to install via conda >> <http://pandas-docs.github.io/pandas-docs-travis/install.html#installing-pan…> from >> our development channel. Builds for osx-64,linux-64,win-64 for Python 2.7 >> and Python 3.5 are all available. >> >> conda install pandas=v0.18.0rc1 -c pandas >> >> Thanks to all who made this release happen. It is a very large release! >> >> Jeff >> >>

1 0

Re: [Pandas-dev] On bug-fix releases and maintenance branches
by P C Feb. 23, 2016

Feb. 23, 2016

Apologies if the formatting for this is messed up, I have just joined the list but would like to reply to a thread that started before I joined, so I have to copy and paste relevant items in. Specifically, I have comments on this: - What are other people's experiences with upgrading pandas? And would more bug-fix releases actually ease the upgrading? I work for a large company with an established Python code base (3+ years of maintenance, tens of users). Pandas and numpy are heavily used throughout the code base. In our experience, upgrading pandas is usually quite an involved exercise. When we upgraded from 0.15.2 to 0.17, it took several passes, as follows: * attempted to upgrade from 0.15.2 to 0.16.1: Found the following bugs, raised the following PRs https://github.com/pydata/pandas/issues/10181 / https://github.com/pydata/pandas/pull/10188 https://github.com/pydata/pandas/issues/10195 / https://github.com/pydata/pandas/pull/10196 https://github.com/pydata/pandas/issues/10193 / https://github.com/pydata/pandas/pull/10379 * The above fixes went into 0.16.2 and 0.17.0 so we decided to wait for 0.17.0. * Tried 0.17.0: Found https://github.com/pydata/pandas/issues/11372 and https://github.com/pydata/pandas/issues/11370 * https://github.com/pydata/pandas/issues/11372 was fixed in 0.17.1 but not https://github.com/pydata/pandas/issues/11370 (targeted 0.18.0) * In the end we went with an intermediate build: 0.17.1.post78+g158e85a in the normal git versioning parlance. ( https://github.com/pydata/pandas/pull/11427 was the specific item that we needed- to fix issue 11372 above). We chose not to wait for 0.18.0 because that contains possibly breaking changes to resample. We judged that it was better to go with a known non-official intermediate build of master, that passed all our tests, than to risk another cycle of chasing bugs. We found that from 0.15.2 to 0.17.1, each release fixed one or more issues that affected us but also introduced other issues. I describe this just to add a data point and relate our experience with upgrading, rather than to shout for massive changes in the release process.The main things that hit us were edge cases breaking when core classes were refactored. Thanks, PC

1 0

google summer of code
by Jeff Reback Feb. 15, 2016

Feb. 15, 2016

So I started an application I need to have a co-adminstrator (or several), pls raise your hand if you'd like to do that. I think I tried to have it send e-mail but not sure if it went thru. I also need to know whom could possibly serve as a mentor. Here is a start of an ideas list, needs to be more text / fleshed out. Feel free to add / update. https://github.com/pydata/pandas/wiki/Google-Summer-of-Code The application is due friday the 19th (next FRIDAY)! so pls let me know ASAP about the above Jeff

4 5

ANN: pandas v0.18.0rc1 - RELEASE CANDIDATE
by Jeff Reback Feb. 14, 2016

Feb. 14, 2016

Hi, I'm pleased to announce the availability of the first release candidate of Pandas 0.18.0. Please try this RC and report any issues here: Pandas Issues <https://github.com/pydata/pandas/issues> We will be releasing officially in 1-2 weeks or so. **RELEASE CANDIDATE 1** This is a major release from 0.17.1 and includes a small number of API changes, several new features, enhancements, and performance improvements along with a large number of bug fixes. We recommend that all users upgrade to this version. Highlights include: - pandas >= 0.18.0 will no longer support compatibility with Python version 2.6 GH7718 <https://github.com/pydata/pandas/issues/7718> or version 3.3 GH11273 <https://github.com/pydata/pandas/issues/11273> - Moving and expanding window functions are now methods on Series and DataFrame similar to .groupby like objects, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> . - Adding support for a RangeIndex as a specialized form of the Int64Index for memory savings, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> . - API breaking .resample changes to make it more .groupby like, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> - Removal of support for positional indexing with floats, which was deprecated since 0.14.0. This will now raise a TypeError, see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> - The .to_xarray() function has been added for compatibility with the xarray package <http://xarray.pydata.org/en/stable/> see here <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> . - Addition of the .str.extractall() method <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…>, and API changes to the the .str.extract() method <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…>, and the .str.cat() method <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#whatsnew-0180…> - pd.test() top-level nose test runner is available GH4327 <https://github.com/pydata/pandas/issues/4327> See the Whatsnew <http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html> for much more information. Best way to get this is to install via conda <http://pandas-docs.github.io/pandas-docs-travis/install.html#installing-pan…> from our development channel. Builds for osx-64,linux-64,win-64 for Python 2.7 and Python 3.5 are all available. conda install pandas=v0.18.0rc1 -c pandas Thanks to all who made this release happen. It is a very large release! Jeff

1 0

Fwd: [pydata] Google Summer of Code 2016, GSoC2016
by Jeff Reback Feb. 12, 2016

Feb. 12, 2016

Anyone have interest in being a mentor for google summer of code? I think could put in the application pretty easily (though due next week): https://developers.google.com/open-source/gsoc/timeline if you do pls respond. I would say that we would need at least 1 other person (I can be a mentor). Jeff ---------- Forwarded message ---------- From: Raniere Silva <raniere(a)ime.unicamp.br> Date: Mon, Feb 8, 2016 at 8:53 AM Subject: [pydata] Google Summer of Code 2016, GSoC2016 To: pydata(a)googlegroups.com Hi all, Since Pandas is a NumFOCUS sponsored project and NumFOCUS will apply to be a mentoring organization on GSoC I want to know (1) if Pandas is planning to apply this year and (2) if want to apply under NumFOCUS umbrella. Pandas is welcome and encouraged to apply as separate mentoring organizations directly with Google. We're happy to help you fill out your application and improve your ideas pages, as well as link your page to help students find you. We may also be able to be a reference for you. It is totally fine if you want to use the NumFOCUS umbrella org as a backup plan in case you don't get selected and we do! Cheers, Raniere -- You received this message because you are subscribed to the Google Groups "PyData" group. To unsubscribe from this group and stop receiving emails from it, send an email to pydata+unsubscribe(a)googlegroups.com. For more options, visit https://groups.google.com/d/optout.

1 0