Mailman 3 March 2019 - Pandas-dev

Python library to extract tables from PDF files
by Vinayak Mehta April 30, 2019

April 30, 2019

Hello everyone! I'm a software engineer based out of New Delhi, India. I've been a long time user and have used it in countless projects and scripts! Thanks to the core developers and contributors for working on it! I recently released a Python library which lets users extract data tables out of PDF files, my first open-source library! Here's the link: https://github.com/socialcopsdev/camelot It has a similar API to the pandas read_* functions, bearing most similarity to read_html(). Like read_html(), it has a read_pdf() main interface which returns a list of pandas DataFrames for each table found in the PDF file, and contains two flavors for parsing different types of tables! I've created a comparison with other open-source PDF table extraction libraries and tools in the wiki here <https://github.com/socialcopsdev/camelot/wiki/Comparison-with-other-PDF-Tab…> . I would be really grateful if you could check it out and see if its useful to you, and give me any feedback that may help me improve it, I promise if would take less than 5 minutes of your time! :) To the core devs: I was wondering if pandas would be open to accept this library as a contribution to its read_* interface? The library uses OpenCV's morphological transformations to detect lines in PDFs when flavor='lattice', which I could vendorize or re-implement. It also has two system specific dependencies which are python-tk (used by matplotlib) and ghostscript (used to convert PDF to PNG). The first one shouldn't pose a problem since pandas also uses matplotlib, and for the second one, I could look for a Python library alternative to ghostscript. Looking forward to hearing from you all! Thanks for your time! Vinayak

2 4

StaticFrame
by Chris Bartak March 22, 2019

March 22, 2019

Maybe others have seen this, I just heard about it on the Talk Python podcast <https://talkpython.fm/episodes/show/204/staticframe-like-pandas-but-safer> the other day, thought I'd share. https://github.com/InvestmentSystems/static-frame https://static-frame.readthedocs.io/en/latest/intro.html Some interesting API decisions - columns are immutable, arguably cleaned up indexing/iteration. Maybe some ideas to borrow.

2 1

ANN: Pandas 0.24.2 released
by Joris Van den Bossche March 15, 2019

March 15, 2019

Hi, I'm pleased to announce pandas 0.24.2, released earlier this week. This is a minor bug-fix release in the 0.24.x series and includes some regression fixes and bug fixes. We recommend that all users upgrade to this version. See the full whatsnew <http://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.2.html> for a list of all the changes. The release can be installed with conda from the defaults and conda-forge channels: conda install pandas Or via PyPI: python -m pip install --upgrade pandas A total of 25 people contributed patches to this release. People with a “+” by their names contributed a patch for the first time. - Albert Villanova del Moral - Arno Veenstra + - chris-b1 - Devin Petersohn + - EternalLearner42 + - Flavien Lambert + - gfyoung - Gioia Ballin - jbrockmendel - Jeff Reback - Jeremy Schendel - Johan von Forstner + - Joris Van den Bossche - Josh - Justin Zheng - Kendall Masse - Matthew Roeschke - Max Bolingbroke + - rbenes + - Sterling Paramore + - Tao He + - Thomas A Caswell - Tom Augspurger - Vibhu Agarwal + - William Ayd - Zach Angell

1 0

Pandas at PyCon Maintainers Summit
by Tom Augspurger March 7, 2019

March 7, 2019

PyCon is having a Maintainers Summit[1] on Saturday, May 4th (the second day of PyCon). Are any of the maintainers interested in submitting something to their CFP (I'm not attending this year)? Proposals are due March 15th. Tom [1]: https://www.papercall.io/pycon-maintainers-summit

2 1

Need help for contributing
by Kapil Patel March 6, 2019

March 6, 2019

Hi, I am a software engineer, trying my hands to learn DataScience. I came across a typo error/grammatical error in the article http://pandas.pydata.org/pandas-docs/stable/getting_started/10min.html Can you please tell me if codebase of this is hosted on GitHub so I can make changes and open a PR request? I tried to find and found this https://github.com/pandas-dev/pandas-website but I am not sure if this GitHub refers to the said website documentation. -Kapil Patel Software Engineer BE(CSE) Indore, India kapilepatel(a)gmail.com +91 881 756 3996

2 2

Remaining API changes / deprecations to do before 1.0 (wishlist)
by Joris Van den Bossche March 5, 2019

March 5, 2019

Hi all, In the summer we had the following plan: do remaining deprecations / changes for 0.24, stabilize this in 0.25, and then clean-up deprecations in 1.0. But, we need to re-evaluate if we are still on schedule for this plan. Because if there are still remaining deprecations or API changes we certainly want to do before 1.0 (which is likely I think), we probably consider still doing a couple of such changes in 0.25 (with or without an additional 0.26 between 0.25 and 1.0). But to be able to make a better idea about what we still want to do, I think it would be good to try to come up with a list of items that we would *ideally* like to do before calling it a 1.0. That should give a better idea on how long a 0.25 might take, if we want a 0.26 before doing 1.0, etc. *So, what is your whishlist of bigger API changes / deprecations you still want to do before 1.0?* What I think we are looking for are the "bigger" topics (because of their user impact and/or impossibility to do it cleanly with a warning). For example, the deprecation of a couple of esoteric functions like Series.nonzero is nice to do now, but would also not be a huge burden to deprecate it after 1.0 and keep that warning throughout 1.x. So this would certainly not be a blocker for 1.0. On the other hand, topics like deprecating the inplace keyword, or changing the return value when indexing into an index with duplicates (#9519 <https://github.com/pandas-dev/pandas/issues/9519>), deprecation of SparseDataFrame, ... to name a few, those have a much bigger impact, and could potentially be seen as a blocker (or at least an issue to try to focus on). And further, much bigger things like complete refactor of the BlockManager, are then again topics clearly out of scope for 1.0 So it the middle category kind of issues that I think we should try to list. The list will probably be too big, but if we have such a list then we can at least have a basis for discussion and try to prioritize / identify some actual blockers. Cheers Joris

3 4

Next pandas dev hangout
by Tom Augspurger March 5, 2019

March 5, 2019

Hi everyone, It'd be good to have another dev chat this week to make sure we're on track for the next couple of releases. Our usual time is Thursday at 18:00 UTC (10:00 Pacific, 13:00 Eastern, 19:00 Central Europe). Does Thursday, March 7th at 18:00 UTC work for people? Is there another day / time that would be better? I'll start to collect agenda items in our minutes <https://docs.google.com/document/d/1tGbTiYORHiSPgVMXawiweGJlBw5dOkVJLY-lico…>. Others can add items or email me with things they would like to discuss. Tom

6 6