Mailman 3 April 2013 - Pandas-dev

datetimes
by Jeff Reback May 13, 2013

May 13, 2013

Currently we allow ONLY datetime64[ns] as an internal representation (and analgously timedelta64[ns] for timedeltas). There are several issues where things like this are done: a) Series([np.datetime(2013,1,1),np.datetime(2013,1,2)],dtype='M8[ms]') b) Series([datetime(2013,1,1),datetime(2013,1,2)],dtype='M8[D]') in a) the np.datetimes are by default [us], so we need to do a conversion to M8[ns], ok, can do that to keep the internal rep, but what about the dtype specified? is this effectively an astype, but then is this conceptually just a display thing, e.g. the user wants to view the data as [ms], rather than [ns] several options to think about: 1) ignore completely the passed dtype and do some conversions on np.datetime64 (which we already do) to guarantee a M8[ns] internally (we do this now, but bork on a passed dtype that is not M8[ns] when the data is M8) 2) keep the passed dtype (or the inferred dtype) internally, effectively making datetimes a suite of M8[ms,D,s,ns......] 3) keep data a M8[ns] internally and provide an asfreq which works kind of like the PeriodIndex method, which can provide a DateTimeIndex I guess of the requested frequency? but then I keep thinking, is there any actual difference between 20130101 15:00:01.12345 in [ms], or [ns] (right now no) Any thoughts....I know I am ramblings a bit, but confused over what is even necessary here... Jeff

2 2

Biggest Fake Conference in Computer Science
by eliswilson＠hushmail.com April 30, 2013

April 30, 2013

Biggest Fake Conference in Computer Science We are researchers from different parts of the world and conducted a study on the world’s biggest bogus computer science conference WORLDCOMP http://sites.google.com/site/worlddump1 organized by Prof. Hamid Arabnia from University of Georgia, USA. We submitted a fake paper to WORLDCOMP 2011 and again (the same paper with a modified title) to WORLDCOMP 2012. This paper had numerous fundamental mistakes. Sample statements from that paper include: (1). Binary logic is fuzzy logic and vice versa (2). Pascal developed fuzzy logic (3). Object oriented languages do not exhibit any polymorphism or inheritance (4). TCP and IP are synonyms and are part of OSI model (5). Distributed systems deal with only one computer (6). Laptop is an example for a super computer (7). Operating system is an example for computer hardware Also, our paper did not express any conceptual meaning. However, it was accepted both the times without any modifications (and without any reviews) and we were invited to submit the final paper and a payment of $500+ fee to present the paper. We decided to use the fee for better purposes than making Prof. Hamid Arabnia richer. After that, we received few reminders from WORLDCOMP to pay the fee but we never responded. This fake paper is different from the two fake papers already published (see https://sites.google.com/site/worlddump4 for details) in WORLDCOMP. We MUST say that you should look at the above website if you have any thoughts of participating in WORLDCOMP. DBLP and other indexing agencies have stopped indexing WORLDCOMP’s proceedings since 2011 due to its fakeness. See http://www.informatik.uni-trier.de/~ley/db/conf/icai/index.html for of one of the conferences of WORLDCOMP and notice that there is no listing after 2010. See Section 2 of http://sites.google.com/site/dumpconf for comments from well-known researchers about WORLDCOMP. The status of your WORLDCOMP papers can be changed from scientific to other (i.e., junk or non-technical) at any time. Better not to have a paper than having it in WORLDCOMP and spoil the resume and peace of mind forever! Our study revealed that WORLDCOMP is money making business, using University of Georgia mask, for Prof. Hamid Arabnia. He is throwing out a small chunk of that money (around 20 dollars per paper published in WORLDCOMP’s proceedings) to his puppet (Mr. Ashu Solo or A.M.G. Solo) who publicizes WORLDCOMP and also defends it at various forums, using fake/anonymous names. The puppet uses fake names and defames other conferences to divert traffic to WORLDCOMP. He also makes anonymous phone calls and threatens the critiques of WORLDCOMP (See Item 7 of Section 5 of above website). That is, the puppet does all his best to get a maximum number of papers published at WORLDCOMP to get more money into his (and Prof. Hamid Arabnia’s) pockets. Prof. Hamid Arabnia makes a lot of tricks. For example, he appeared in a newspaper to fool the public, claiming him a victim of cyber-attack (see Item 8 in Section 5 of above website). Monte Carlo Resort (the venue of WORLDCOMP for more than 10 years, until 2012) has refused to provide the venue for WORLDCOMP’13 because of the fears of their image being tarnished due to WORLDCOMP’s fraudulent activities. That is why WORLDCOMP’13 is taking place at a different resort. WORLDCOMP will not be held after 2013. The draft paper submission deadline is over but still there are no committee members, no reviewers, and there is no conference Chairman. The only contact details available on WORLDCOMP’s website is just an email address! We ask Prof. Hamid Arabnia to publish all reviews for all the papers (after blocking identifiable details) since 2000 conference. Reveal the names and affiliations of all the reviewers (for each year) and how many papers each reviewer had reviewed on average. We also ask him to look at the Open Challenge (Section 6) at https://sites.google.com/site/moneycomp1 and respond if he has any professional values. Sorry for posting to multiple lists. Spreading the word is the only way to stop this bogus conference. Please forward this message to other mailing lists and people. We are shocked with Prof. Hamid Arabnia and his puppet’s activities at http://worldcomp-fake-bogus.blogspot.com Search Google using the keyword worldcomp fake for additional links.

1 0

Setuptools detection of host cpuinfo and/or header file availablity
by Stephen Lin April 27, 2013

April 27, 2013

Hi Yoval, I had a question about setuptools that Wes suggested I run by you...do you know if there's there's a canonical (and, hopefully by extension, reliable and cross-platform...) way of checking for host cpu capabilities and header file availability in a python setuptools script to configure a build? This is for potentially including code using SSE intrinsics into pandas. Stephen

3 8

0.11.x maintenance
by Wes McKinney April 26, 2013

April 26, 2013

hey folks, what is the story with 0.12.x? Are we breaking any APIs yet? If so we should probably plan for a 0.11.1 that is *bugfix *only and we somewhat diligent about backporting critical bugfixes. Let me know what you think. - Wes

3 4

0.11 highlights?
by Wes McKinney April 23, 2013

April 23, 2013

Could I get a 2-paragraph summary of 0.11 highlights for the release e-mail? I'm going to make sure Windows is good, take a look at the two parser issues, and then look to cut the release in the next 48 hours. Any pressing concerns? thanks, Wes

3 7

pickle is evil
by Jeff Reback April 23, 2013

April 23, 2013

I thought I'd share a particularly evil pickle issue. In my refactor of Series to not subclass ndarray, the new pickling tests were breaking. No suprise because I changed __getstate__ to pickle via the BlockManager. In order to ensure compat I thought I could just fix __setstate__ and figure out what to do based on the return state (e.g. the len of the state returned as a tuple or dict or whatever). But no...apparently the reconstruction algorithm takes the class name that it see and tries to create it w/o using __new__ (or anything else that you can intercept), it uses a builtin method called _reconstruct (which is a builtin, but I can't figure out how to override it at all, must be only c-code). And then numpy gets ahold of it (as its an extension type), and complains becuase the class I am trying to instantiate actually isn't a sub-class of ndarray (which it pre-supposes). So, a bit hacky, but using a custom unpickler, then matching on a compatbility class (that sub-classes from ndarray), allows me to return the correct class. The good thing here is that this whole routine isn't even called unless there is a TypeError on the original unpickle whoosh! -------- # new module: compat/unpickle_compat.py import numpy as np import pandas from pandas.core.series import Series from pandas.sparse.series import SparseSeries import pickle class Unpickler(pickle.Unpickler): pass def load_reduce(self): stack = self.stack args = stack.pop() func = stack[-1] if type(args[0]) is type: n = args[0].__name__ if n == 'DeprecatedSeries': stack[-1] = object.__new__(Series) return elif n == 'DeprecatedSparseSeries': stack[-1] = object.__new__(SparseSeries) return value = func(*args) stack[-1] = value Unpickler.dispatch['R'] = load_reduce def load(file): # try to load a compatibility pickle # fake the old class hierarchy # if it works, then return the new type objects try: pandas.core.series.Series = DeprecatedSeries pandas.sparse.series.SparseSeries = DeprecatedSparseSeries with open(file,'rb') as fh: return Unpickler(fh).load() except: raise finally: pandas.core.series.Series = Series pandas.sparse.series.SparseSeries = SparseSeries class DeprecatedSeries(Series, np.ndarray): pass class DeprecatedSparseSeries(DeprecatedSeries): pass

2 4

0.10, 0.10.1 on pydata page
by Jeff Reback April 23, 2013

April 23, 2013

prob should update the website whenever u have a chance for these versions

2 1

docs
by Jeff Reback April 22, 2013

April 22, 2013

Wes after release is it possible to do a doc update at some point? (ok the released docs) if so just merge into master and ten u build docs and update? (like u do now for dev) or is this more trouble than worth? I just have 1 specifc in mind, but won't get to it for a bit

3 2

Congratulations! You may have already saved 8 minutes per travis build.
by yoval p. April 17, 2013

April 17, 2013

Hi, I just merged PR #3383, which allows whitelisted forks to use a network cache server for speeding up travis. All commit-bit bearing persons have has their fork whitelisted. The upstream pydata repo is also whitelisted, naturally. Proving that often all you really need to do is ask nicely, if you include the magic incantation "PLEASE_TRAVIS_FASTER" (just "PTF" also works) anywhere in the commit message, the travis build time should drop significantly, to under 4 minutes in the common case. Example: https://travis-ci.org/y-p/pandas/builds/6416089 Cheers, y-p

2 1

Fwd: [Pytables-users] ANN: numexpr 2.1 RC1 available!
by Jeff Reback April 15, 2013

April 15, 2013

when this is final will want to pick this up in the 3.2 full dep Travis build (if its not already) Begin forwarded message: > From: Francesc Alted <faltet(a)gmail.com> > Date: April 14, 2013, 4:19:31 PM EDT > To: numexpr(a)googlegroups.com > Cc: Discussion list for PyTables <pytables-users(a)lists.sourceforge.net> > Subject: [Pytables-users] ANN: numexpr 2.1 RC1 available! > Reply-To: Discussion list for PyTables <pytables-users(a)lists.sourceforge.net> > > ============================ > Announcing Numexpr 2.1RC1 > ============================ > > Numexpr is a fast numerical expression evaluator for NumPy. With it, > expressions that operate on arrays (like "3*a+4*b") are accelerated > and use less memory than doing the same calculation in Python. > > It wears multi-threaded capabilities, as well as support for Intel's > VML library, which allows for squeezing the last drop of performance > out of your multi-core processors. > > What's new > ========== > > This version adds compatibility for Python 3. A bunch of thanks to > Antonio Valentino for his excelent work on this.I apologize for taking > so long in releasing his contributions. > > In case you want to know more in detail what has changed in this > version, see: > > http://code.google.com/p/numexpr/wiki/ReleaseNotes > > or have a look at RELEASE_NOTES.txt in the tarball. > > Where I can find Numexpr? > ========================= > > The project is hosted at Google code in: > > http://code.google.com/p/numexpr/ > > This is a release candidate 1, so it will not be available on the PyPi > repository. I'll post it there when the final version will released. > > Share your experience > ===================== > > Let us know of any bugs, suggestions, gripes, kudos, etc. you may > have. > > > Enjoy! > > -- > Francesc Alted > > > ------------------------------------------------------------------------------ > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > _______________________________________________ > Pytables-users mailing list > Pytables-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pytables-users

2 1