Mailman 3 August 2022 - Pandas-dev

PDEPs: pandas enhancement proposals
by Marc Garcia Nov. 30, 2022

Nov. 30, 2022

We're in the process of implementing PDEPs, equivalent to Python's PEPs and NumPy's NEPs, but for pandas. This should help build the roadmap, make discussions more efficient, obtain more structured feedback from the community, and add visibility to agreed future plans for pandas. The initial implementation (workflow) is a bit simpler than PEP or NEP, but we'll iterate in the future as convenient. You can see the PR for PDEP-1 with the purpose, scope and guidelines here: https://github.com/pandas-dev/pandas/pull/47444 Feedback is very welcome.

6 10

August 2022 monthly community meeting (Wednesday August 10, UTC 18:00)
by Joris Van den Bossche Nov. 8, 2022

Nov. 8, 2022

Hi all, A reminder that the next monthly dev call is tomorrow (Wednesday, August 10) at 18:00 UTC (1pm Central). Our calendar is at https://pandas.pydata.org/docs/development/meeting.html#calendar to check your local time. All are welcome to attend! Video Call: https://us06web.zoom.us/j/84484803210?pwd=TjUxNmcyNHcvcG9SNGJvbE53Y21GZz09 Minutes: https://docs.google.com/document/u/1/d/1tGbTiYORHiSPgVMXawiweGJlBw5dOkVJLY-… Joris

1 3

pandas ASVs
by Richard Shadrach Aug. 30, 2022

Aug. 30, 2022

I noticed a bit ago that the ASVs on the pandas webpage were no longer being updated. https://pandas.pydata.org/speed/pandas/ Tom shared with me his setup and gave me permissions to https://github.com/asv-runner (thanks Tom!). I now have a machine dedicated to running the ASVs and updating the site here: https://asv-runner.github.io/asv-collection/pandas/ I plan to be going through the regressions identified there and notifying PRs as necessary; manually at first but then looking into a more automated solution. I also plan to port Tom's work in asv-runner over to Docker. >From Marc, it sounds like there might be some changes coming to the webserver, so I think it's best to wait until that dust settles before getting the ASVs on the pandas webpage to auto-update. Best, Richard

2 3

Re: [Pandas-dev] [CrowdStrike/falconpy] Why I recieved this output: (Discussion #760)
by נתי שטרן Aug. 25, 2022

Aug. 25, 2022

בתאריך יום חמישי, 25 באוגוסט 2022, מאת Joshua Hiller < notifications(a)github.com>: > Hi @NSH531 <https://github.com/NSH531> - > > If passing an index doesn't help, you can probably get more assistance > from the Pandas issue board located here: https://github.com/pandas-dev/ > pandas/issues > > — > Reply to this email directly, view it on GitHub > <https://github.com/CrowdStrike/falconpy/discussions/760#discussioncomment-3…>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABNBJWNTMFYE4FBSUJ5FGM3V2…> > . > You are receiving this because you were mentioned.Message ID: > <CrowdStrike/falconpy/repo-discussions/760/comments/3475675(a)github.com> > -- <https://netanel.ml>

1 0

Re: [Pandas-dev] [CrowdStrike/falconpy] Why I recieved this output: (Discussion #760)+pandas DataFrame error
by נתי שטרן Aug. 25, 2022

Aug. 25, 2022

2022-08-25 17:53 GMT‎+03:00‎, נתי שטרן <nsh531(a)gmail.com>: > Someone here is understanding at pandas? > > בתאריך יום חמישי, 25 באוגוסט 2022, מאת Joshua Hiller < > notifications(a)github.com>: > >> This is pretty far out of scope, but reviewing the error message in your >> IDE it appears you are passing scalars to the Dataframe. Since there is >> no >> index, you're given an error. Googling about, several folks who hit this >> error pass the keyword index=[0] and provide a dummy index. Depending on >> your goals, this may not be the correct solution. >> >> — >> Reply to this email directly, view it on GitHub >> <https://github.com/CrowdStrike/falconpy/discussions/760#discussioncomment-3…>, >> or unsubscribe >> <https://github.com/notifications/unsubscribe-auth/ABNBJWOHTGWD54YFK4D4MQTV2…> >> . >> You are receiving this because you authored the thread.Message ID: >> <CrowdStrike/falconpy/repo-discussions/760/comments/3475300(a)github.com> >> > > > -- > <https://netanel.ml> > -- <https://netanel.ml>

1 1

Plans of future PDEPs
by Marc Garcia Aug. 25, 2022

Aug. 25, 2022

I'm planning to work on few PDEPs, and I think it's probably worth sharing the general ideas in advance, in case anyone has early feedback, wants to coordinate, or it's simply useful for others to know before the draft PDEPs are ready. Not intending to open the discussion on the details of them here, which it probably makes more sense to discuss in the draft on the PDEP when ready. *Governance*: We discussed about defining exactly how a PDEP is approved while implementing them, and it's probably useful to clarify the project decision making in general. And also, I want to propose the creation of some workgroups within pandas to be more efficient managing things. Define their roles, how members are selected... Besides the existing CoC and NumFOCUS (finances) workgroups, I think it could be useful to have workgroups for the infrastructure (manage access to our servers, the github org, the distribution lists...), for the communications (tweeting, blogging, coordinate with NumFOCUS and other projects...). Maybe a release workgroup, and a steering committee if people find them useful... *Release*: I think it would be useful to have a PDEP detailing the release process. Similar to our policies doc <https://pandas.pydata.org/docs/development/policies.html>, but being more detailed and explicit on when exactly releases happen, and what's the criteria. Not necessarily change the current release policy, but I think we can better manage expectations of both users and developers, and be more efficient in the discussions if there is less ambiguity on when releases are going to happen. *IO modules as plugins*: The idea would be to create a framework to extend pandas with IO functions, and move some of the current IO adaptors (e.g. stata, spss, feather...) to third-party projects. I think this would encourage people to build more pandas IO packages (for new formats, or improved versions of the existing), it would make these modules better maintained (easier to become a maintainer, maintainers experts in the formats...), and would reduce the complexity of pandas itself, the CI, reduce significantly the number of optional dependencies, and offer users a more natural way to install things (install pandas-sas, as opposed to have read_sas failing and having to install its dependencies). *Duplicate columns*: pandas currently supports it, but it's not clear if there are many (or any) use cases where this is helpful, and this clearly makes pandas more complex, in its internals, and in its usability (e.g. almost all users would expect `df['my_col']` to return a Series, but no necessarily true, since there can be many columns "my_col", and it could return a DataFrame). I think it's useful to create a PDEP with advantages and actual use cases of having duplicate columns, and its cost in terms of problems and extra complexity. And make a decision on whether we want to continue supporting it. Cheers,

2 1

ANN: pandas 1.5.0 RC0
by Marc Garcia Aug. 25, 2022

Aug. 25, 2022

We are happy to announce the *release candidate* of pandas 1.5.0. It can be installed from our conda-forge and PyPI packages via mamba, conda and pip, for example: mamba install -c conda-forge/label/pandas_rc pandas==1.5.0rc0 python -m pip install --upgrade --pre pandas==1.5.0rc0 Users having pandas code in production and maintainers of libraries with pandas as a dependency are *strongly* recommended to run their test suites with the release candidate, and report any breaking change to our issue tracker <https://github.com/pandas-dev/pandas/issues/new/choose> before the official 1.5.0 release. You can find the documentation of pandas 1.5.0 here <https://pandas.pydata.org/pandas-docs/version/1.5.0rc0/index.html>, and the list of changes in 1.5.0, in the release notes page <https://pandas.pydata.org/pandas-docs/version/1.5.0rc0/whatsnew/v1.5.0.html> . We expect to release the final version of pandas 1.5.0 on September 7th, but the final date will depend on the issues reported to the release candidate.

1 0

Retention of pandas nightly builds
by Marc Garcia Aug. 19, 2022

Aug. 19, 2022

Hi all, Seems like at the moment we're keeping all history of pandas nightly wheels at https://anaconda.org/scipy-wheels-nightly and they are using around 100 Gb storage. Other projects are in general keeping the last 5 days of builds, or in some cases like scikit-learn they just keep the last. I assume there is no reason to keep all history more than not having an automated way to delete old wheels yet. And I personally can't think of a use case to need more than the last build, so if we add a CI job to delete old wheels, my preference would be to just keep overwritting the same files, which I think makes things simpler for us and for users. But maybe other people do have use cases for keeping the last few days of wheels. Does anyone have the need for something like a 5 days retention of nightly pandas wheels? We'll start removing old wheels in couple of days, or once the discussion here has a conclusion. Cheers!

2 1

this is a thank you message
by Tiago Alberto Aug. 11, 2022

Aug. 11, 2022

I really dont know who u folks are, but pandas changed my life forever. Just a student of course, a beginner, but completely conscious about how f&*ing terrifying and extenous job u guys have been doing so far! Tnks for all of that folks... for real.. A Brazilian enthusiast... A., Tiago.

2 1