Mailman 3 January 2015 - Python-Dev

New Windows installer for Python 3.5
by Steve Dower 13 Jan '15

13 Jan '15

I've put together a short post showing where I've been taking the Windows installer for Python 3.5, since I know there are interested people on this list who will have valuable feedback. http://stevedower.id.au/blog/the-python-3-5-installer/ Nothing is merged in yet and everything can still change, so I'm keen to hear whatever feedback people have. I've tried to make improvements fairly for first-time users through to sysadmins, but if I've missed something big I'd like to hear about it before we get too close to alpha 1. Cheers, Steve

11 30

Summary of Python tracker Issues
by Python tracker 09 Jan '15

09 Jan '15

ACTIVITY SUMMARY (2015-01-02 - 2015-01-09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4726 (+20) closed 30244 (+37) total 34970 (+57) Open issues with patches: 2214 Issues opened (44) ================== #14099: ZipFile.open() should not reopen the underlying file http://bugs.python.org/issue14099 reopened by serhiy.storchaka #14853: test_file.py depends on sys.stdin being unseekable http://bugs.python.org/issue14853 reopened by haypo #16192: ctypes - documentation example http://bugs.python.org/issue16192 reopened by berker.peksag #19776: Provide expanduser() on Path objects http://bugs.python.org/issue19776 reopened by haypo #22919: Update PCBuild for VS 2015 http://bugs.python.org/issue22919 reopened by haypo #23152: fstat64 required on Windows http://bugs.python.org/issue23152 opened by steve.dower #23155: unittest: object has no attribute '_removed_tests' http://bugs.python.org/issue23155 opened by wiz #23156: Update tix install information in tkinter tix chapter of doc http://bugs.python.org/issue23156 opened by terry.reedy #23159: argparse: Provide equivalent of optparse.OptionParser.{option_ http://bugs.python.org/issue23159 opened by emcd #23160: Respect the environment variable SVNROOT in external-common.ba http://bugs.python.org/issue23160 opened by anselm.kruis #23162: collections.abc sequences don't check identity before equality http://bugs.python.org/issue23162 opened by Devin Jeanpierre #23163: pdb docs need to contain a statement on threads/multithreaded http://bugs.python.org/issue23163 opened by krichter #23164: "pydoc filter" documentation restrictive http://bugs.python.org/issue23164 opened by lebigot #23166: urllib2 ignores opener configuration under certain circumstanc http://bugs.python.org/issue23166 opened by crazyjurich #23169: Reflect that PreReq and BuildPreReq are deprecated in the late http://bugs.python.org/issue23169 opened by radeksimko #23171: csv.writer.writerow() does not accept generator (must be coerc http://bugs.python.org/issue23171 opened by jdufresne #23173: asyncio: kill the subprocess if the creation failed http://bugs.python.org/issue23173 opened by haypo #23174: shelve.open fails with error "anydbm.error: db type could not http://bugs.python.org/issue23174 opened by krichter #23176: socket.recvfrom(0) waits for data http://bugs.python.org/issue23176 opened by Sworddragon #23177: test_ssl: failures on OpenBSD with LibreSSL http://bugs.python.org/issue23177 opened by haypo #23180: Rename IDLE's "Windows" menu item to "Window" http://bugs.python.org/issue23180 opened by Al.Sweigart #23181: Unicode "code point" should be two words in documentation http://bugs.python.org/issue23181 opened by Al.Sweigart #23182: Update grammar tests to use new style for annotated function d http://bugs.python.org/issue23182 opened by IanLee1521 #23183: timeit CLI best of 3: undocumented output format http://bugs.python.org/issue23183 opened by hachat #23184: Unused imports, variables, file in IDLE http://bugs.python.org/issue23184 opened by Al.Sweigart #23185: add inf and nan to math module http://bugs.python.org/issue23185 opened by ethan.furman #23187: Segmentation fault, possibly asyncio related http://bugs.python.org/issue23187 opened by karamanolev #23188: Exception chaining should trigger for non-normalised exception http://bugs.python.org/issue23188 opened by ncoghlan #23189: Set docstrings to empty string when optimizing with -OO. http://bugs.python.org/issue23189 opened by jvs #23191: fnmatch regex cache use is not threadsafe http://bugs.python.org/issue23191 opened by mschmitzer #23192: Generator return value ignored in lambda function http://bugs.python.org/issue23192 opened by Rosuav #23193: Please support "numeric_owner" in tarfile http://bugs.python.org/issue23193 opened by mvo #23195: Sorting with locale (strxfrm) does not work properly with Pyth http://bugs.python.org/issue23195 opened by pnugues #23197: asyncio: check if a future is cancelled before calling set_res http://bugs.python.org/issue23197 opened by haypo #23198: asyncio: refactor StreamReader http://bugs.python.org/issue23198 opened by haypo #23199: libpython27.a in amd64 release is 32-bit http://bugs.python.org/issue23199 opened by zwelch #23200: Clarify max_length and flush() for zlib decompression http://bugs.python.org/issue23200 opened by vadmium #23201: Decimal(0)**0 is an error, 0**0 is 1, but Decimal(0) == 0 http://bugs.python.org/issue23201 opened by Devin Jeanpierre #23202: pyvenv does not fail like documented when a venv already exist http://bugs.python.org/issue23202 opened by The Compiler #23203: Aliasing import of sub-{module,package} from the package raise http://bugs.python.org/issue23203 opened by franck #23205: Unit test needed for IDLE's GrepDialog.py's findfiles() http://bugs.python.org/issue23205 opened by Al.Sweigart #23206: json.dumps(ensure_ascii=False) is ~10x slower than json.dumps( http://bugs.python.org/issue23206 opened by naoki #23207: logging.basicConfig does not validate keyword arguments http://bugs.python.org/issue23207 opened by The Compiler #23208: asyncio: add BaseEventLoop._current_handle http://bugs.python.org/issue23208 opened by haypo Most recent 15 issues with no replies (15) ========================================== #23207: logging.basicConfig does not validate keyword arguments http://bugs.python.org/issue23207 #23205: Unit test needed for IDLE's GrepDialog.py's findfiles() http://bugs.python.org/issue23205 #23202: pyvenv does not fail like documented when a venv already exist http://bugs.python.org/issue23202 #23199: libpython27.a in amd64 release is 32-bit http://bugs.python.org/issue23199 #23183: timeit CLI best of 3: undocumented output format http://bugs.python.org/issue23183 #23182: Update grammar tests to use new style for annotated function d http://bugs.python.org/issue23182 #23181: Unicode "code point" should be two words in documentation http://bugs.python.org/issue23181 #23173: asyncio: kill the subprocess if the creation failed http://bugs.python.org/issue23173 #23169: Reflect that PreReq and BuildPreReq are deprecated in the late http://bugs.python.org/issue23169 #23164: "pydoc filter" documentation restrictive http://bugs.python.org/issue23164 #23163: pdb docs need to contain a statement on threads/multithreaded http://bugs.python.org/issue23163 #23159: argparse: Provide equivalent of optparse.OptionParser.{option_ http://bugs.python.org/issue23159 #23148: Missing the charset parameter in as_encoded_word() http://bugs.python.org/issue23148 #23147: Possible error in _header_value_parser.py http://bugs.python.org/issue23147 #23146: Incosistency in pathlib between / and \ http://bugs.python.org/issue23146 Most recent 15 issues waiting for review (15) ============================================= #23208: asyncio: add BaseEventLoop._current_handle http://bugs.python.org/issue23208 #23206: json.dumps(ensure_ascii=False) is ~10x slower than json.dumps( http://bugs.python.org/issue23206 #23205: Unit test needed for IDLE's GrepDialog.py's findfiles() http://bugs.python.org/issue23205 #23200: Clarify max_length and flush() for zlib decompression http://bugs.python.org/issue23200 #23198: asyncio: refactor StreamReader http://bugs.python.org/issue23198 #23197: asyncio: check if a future is cancelled before calling set_res http://bugs.python.org/issue23197 #23193: Please support "numeric_owner" in tarfile http://bugs.python.org/issue23193 #23191: fnmatch regex cache use is not threadsafe http://bugs.python.org/issue23191 #23185: add inf and nan to math module http://bugs.python.org/issue23185 #23184: Unused imports, variables, file in IDLE http://bugs.python.org/issue23184 #23182: Update grammar tests to use new style for annotated function d http://bugs.python.org/issue23182 #23181: Unicode "code point" should be two words in documentation http://bugs.python.org/issue23181 #23180: Rename IDLE's "Windows" menu item to "Window" http://bugs.python.org/issue23180 #23173: asyncio: kill the subprocess if the creation failed http://bugs.python.org/issue23173 #23171: csv.writer.writerow() does not accept generator (must be coerc http://bugs.python.org/issue23171 Top 10 most discussed issues (10) ================================= #23185: add inf and nan to math module http://bugs.python.org/issue23185 19 msgs #23086: Add start and stop parameters to the Sequence.index() ABC mixi http://bugs.python.org/issue23086 13 msgs #23201: Decimal(0)**0 is an error, 0**0 is 1, but Decimal(0) == 0 http://bugs.python.org/issue23201 10 msgs #23152: fstat64 required on Windows http://bugs.python.org/issue23152 8 msgs #23187: Segmentation fault, possibly asyncio related http://bugs.python.org/issue23187 8 msgs #22906: PEP 479: Change StopIteration handling inside generators http://bugs.python.org/issue22906 7 msgs #22919: Update PCBuild for VS 2015 http://bugs.python.org/issue22919 7 msgs #23162: collections.abc sequences don't check identity before equality http://bugs.python.org/issue23162 7 msgs #23174: shelve.open fails with error "anydbm.error: db type could not http://bugs.python.org/issue23174 7 msgs #23177: test_ssl: failures on OpenBSD with LibreSSL http://bugs.python.org/issue23177 7 msgs Issues closed (39) ================== #3180: Interrupts are lost during readline PyOS_InputHook processing http://bugs.python.org/issue3180 closed by haypo #14076: sqlite3 module ignores placeholders in CREATE TRIGGER code http://bugs.python.org/issue14076 closed by ghaering #18644: Got ResourceWarning: unclosed file when using test function fr http://bugs.python.org/issue18644 closed by berker.peksag #19548: 'codecs' module docs improvements http://bugs.python.org/issue19548 closed by ncoghlan #20487: Odd words in unittest.mock document. http://bugs.python.org/issue20487 closed by berker.peksag #20606: Operator Documentation Example doesn't work http://bugs.python.org/issue20606 closed by benjamin.peterson #21356: Support LibreSSL (instead of OpenSSL): make RAND_egd optional http://bugs.python.org/issue21356 closed by haypo #21980: Implement `logging.LogRecord.__repr__` http://bugs.python.org/issue21980 closed by python-dev #22038: Implement atomic operations on non-x86 platforms http://bugs.python.org/issue22038 closed by haypo #22428: asyncio: KeyboardInterrupt inside a coroutine causes Attribute http://bugs.python.org/issue22428 closed by haypo #22638: ssl module: the SSLv3 protocol is vulnerable ("POODLE" attack) http://bugs.python.org/issue22638 closed by pitrou #23014: Don't have importlib.abc.Loader.create_module() be optional http://bugs.python.org/issue23014 closed by brett.cannon #23119: Remove unicode specialization from set objects http://bugs.python.org/issue23119 closed by rhettinger #23127: socket.setsockopt() is still broken for multicast TTL and Loop http://bugs.python.org/issue23127 closed by benjamin.peterson #23132: Faster total_ordering http://bugs.python.org/issue23132 closed by rhettinger #23140: InvalidStateError on asyncio subprocess task cancelled http://bugs.python.org/issue23140 closed by haypo #23143: Remove some conditional code in _ssl.c http://bugs.python.org/issue23143 closed by pitrou #23145: regrtest: log test loader errors http://bugs.python.org/issue23145 closed by haypo #23149: Typo in PEP-0008 - "this PEP do not" http://bugs.python.org/issue23149 closed by python-dev #23151: _loggerClass is initialized twice http://bugs.python.org/issue23151 closed by python-dev #23153: Clarify Boolean Clause Results http://bugs.python.org/issue23153 closed by r.david.murray #23154: MSVC 2013 Express needlessly rebuilds code http://bugs.python.org/issue23154 closed by zach.ware #23157: Lib/test/time_hashlib.py doesn't work http://bugs.python.org/issue23157 closed by gregory.p.smith #23158: IDLE's help.txt "corrent" typo http://bugs.python.org/issue23158 closed by terry.reedy #23161: collections.abc.MutableSet missing methods http://bugs.python.org/issue23161 closed by rhettinger #23165: Heap overwrite in Python/fileutils.c:_Py_char2wchar() on 32 bi http://bugs.python.org/issue23165 closed by python-dev #23167: Marshal docs say format version is 3; actual format version is http://bugs.python.org/issue23167 closed by python-dev #23168: test_file2k.py depends on sys.stdin being unseekable http://bugs.python.org/issue23168 closed by haypo #23170: test_uuid test_ifconfig_getnode fails with Temporary failure i http://bugs.python.org/issue23170 closed by r.david.murray #23172: Bad file descriptor error occurs if random library is imported http://bugs.python.org/issue23172 closed by haypo #23175: logging.exception doesn't accept custom exc_info http://bugs.python.org/issue23175 closed by vinay.sajip #23178: csv.reader does not handle BOM http://bugs.python.org/issue23178 closed by r.david.murray #23179: New function proposal: string.from_iterable(iterable [,map_fun http://bugs.python.org/issue23179 closed by r.david.murray #23186: expose the client's list of ciphers during the handshake http://bugs.python.org/issue23186 closed by python-dev #23190: OpenSSL fails building with build.bat http://bugs.python.org/issue23190 closed by pitrou #23194: Antigravity prints osascript errors in OS X Yosemite http://bugs.python.org/issue23194 closed by ned.deily #23196: Greek letters not sorted properly http://bugs.python.org/issue23196 closed by r.david.murray #23204: list.index and rest of list methods disagree if a value is in http://bugs.python.org/issue23204 closed by rhettinger #1687125: cannot catch KeyboardInterrupt when using curses getkey() http://bugs.python.org/issue1687125 closed by haypo

1 0

contributing to multiprocessing
by Davin Potts 09 Jan '15

09 Jan '15

Hi all -- I am interested in making some serious ongoing contributions around multiprocessing. My inspiration, first and foremost, comes from the current documentation for multiprocessing. There is great material there but I believe it is being presented in a way that hinders adoption and understanding. I've taken some initial baby-steps to propose specific changes: http://bugs.python.org/issue22952 http://bugs.python.org/issue23100 The first, issue22952, can reasonably be tackled with a patch like I've submitted. Continuing with patches for issue23100 can also be made to work. I realize that reviewing such patches takes non-trivial time from volunteers yet I'm interested in submitting a series of patches to hopefully make the documentation for multiprocessing much more consistent with other module docs and much more accessible to end users. I don't want to simply create more work for other volunteers -- I'd like to volunteer to reduce / share some of their work as well. Beyond the documentation, there is currently a backlog of 186 issues mentioning multiprocessing, some with patches on offer, some without. I'd like to volunteer my time reviewing and triaging these issues. Hopefully you can already get a sense of my voice on issues from what I wrote in those two issues above. Rather than me simply walking through that backlog, offering comments or encouragement here and there on issues, it makes more sense for me to ask: what is the right way for me to proceed? What is the next step towards me helping triage issues? Is there a bridge-keeper with at least three, no more than five questions for me? Thanks, Davin

4 4

datetime nanosecond support (ctd?)
by Matthieu Bec 08 Jan '15

08 Jan '15

newbie first post on this list, if what follows is of context ... Hi all, I'm struggling with issue per the subject, read different threads and issue http://bugs.python.org/issue15443 that started 2012 still opened as of today. Isn't there a legitimate case for nanosecond support? it's all over the place in 'struct timespec' and maybe wrongly I always found python and C were best neighbors. That's for the notional aspect. More practically, aren't we close enough yet with current hardware, PTP and the likes, this deserves more consideration? Maybe this has been mentioned before but the limiting factor isn't just getting nanoseconds, but anything sub-microseconds wont work with the current format. OpcUA that I was looking right now has 10-th us resolution, so really cares about 100ns, but the datetime 1us simply wont cut it. Regards, Matthieu

16 30

My thinking about the development process
by Brett Cannon 08 Jan '15

08 Jan '15

This is a bit long as I espoused as if this was a blog post to try and give background info on my thinking, etc. The TL;DR folks should start at the "Ideal Scenario" section and read to the end. P.S.: This is in Markdown and I have put it up at https://gist.github.com/brettcannon/a9c9a5989dc383ed73b4 if you want a nicer formatted version for reading. # History lesson Since I signed up for the python-dev mailing list way back in June 2002, there seems to be a cycle where we as a group come to a realization that our current software development process has not kept up with modern practices and could stand for an update. For me this was first shown when we moved from SourceForge to our own infrastructure, then again when we moved from Subversion to Mercurial (I led both of these initiatives, so it's somewhat a tradition/curse I find myself in this position yet again). And so we again find ourselves at the point of realizing that we are not keeping up with current practices and thus need to evaluate how we can improve our situation. # Where we are now Now it should be realized that we have to sets of users of our development process: contributors and core developers (the latter whom can play both roles). If you take a rough outline of our current, recommended process it goes something like this: 1. Contributor clones a repository from hg.python.org 2. Contributor makes desired changes 3. Contributor generates a patch 4. Contributor creates account on bugs.python.org and signs the [contributor agreement](https://www.python.org/psf/contrib/contrib-form/) 4. Contributor creates an issue on bugs.python.org (if one does not already exist) and uploads a patch 5. Core developer evaluates patch, possibly leaving comments through our [custom version of Rietveld](http://bugs.python.org/review/) 6. Contributor revises patch based on feedback and uploads new patch 7. Core developer downloads patch and applies it to a clean clone 8. Core developer runs the tests 9. Core developer does one last `hg pull -u` and then commits the changes to various branches I think we can all agree it works to some extent, but isn't exactly smooth. There are multiple steps in there -- in full or partially -- that can be automated. There is room to improve everyone's lives. And we can't forget the people who help keep all of this running as well. There are those that manage the SSH keys, the issue tracker, the review tool, hg.python.org, and the email system that let's use know when stuff happens on any of these other systems. The impact on them needs to also be considered. ## Contributors I see two scenarios for contributors to optimize for. There's the simple spelling mistake patches and then there's the code change patches. The former is the kind of thing that you can do in a browser without much effort and should be a no-brainer commit/reject decision for a core developer. This is what the GitHub/Bitbucket camps have been promoting their solution for solving while leaving the cpython repo alone. Unfortunately the bulk of our documentation is in the Doc/ directory of cpython. While it's nice to think about moving the devguide, peps, and even breaking out the tutorial to repos hosting on Bitbucket/GitHub, everything else is in Doc/ (language reference, howtos, stdlib, C API, etc.). So unless we want to completely break all of Doc/ out of the cpython repo and have core developers willing to edit two separate repos when making changes that impact code **and** docs, moving only a subset of docs feels like a band-aid solution that ignores the big, white elephant in the room: the cpython repo, where a bulk of patches are targeting. For the code change patches, contributors need an easy way to get a hold of the code and get their changes to the core developers. After that it's things like letting contributors knowing that their patch doesn't apply cleanly, doesn't pass tests, etc. As of right now getting the patch into the issue tracker is a bit manual but nothing crazy. The real issue in this scenario is core developer response time. ## Core developers There is a finite amount of time that core developers get to contribute to Python and it fluctuates greatly. This means that if a process can be found which allows core developers to spend less time doing mechanical work and more time doing things that can't be automated -- namely code reviews -- then the throughput of patches being accepted/rejected will increase. This also impacts any increased patch submission rate that comes from improving the situation for contributors because if the throughput doesn't change then there will simply be more patches sitting in the issue tracker and that doesn't benefit anyone. # My ideal scenario If I had an infinite amount of resources (money, volunteers, time, etc.), this would be my ideal scenario: 1. Contributor gets code from wherever; easiest to just say "fork on GitHub or Bitbucket" as they would be official mirrors of hg.python.org and are updated after every commit, but could clone hg.python.org/cpython if they wanted 2. Contributor makes edits; if they cloned on Bitbucket or GitHub then they have browser edit access already 3. Contributor creates an account at bugs.python.org and signs the CLA 3. The contributor creates an issue at bugs.python.org (probably the one piece of infrastructure we all agree is better than the other options, although its workflow could use an update) 4. If the contributor used Bitbucket or GitHub, they send a pull request with the issue # in the PR message 5. bugs.python.org notices the PR, grabs a patch for it, and puts it on bugs.python.org for code review 6. CI runs on the patch based on what Python versions are specified in the issue tracker, letting everyone know if it applied cleanly, passed tests on the OSs that would be affected, and also got a test coverage report 7. Core developer does a code review 8. Contributor updates their code based on the code review and the updated patch gets pulled by bugs.python.org automatically and CI runs again 9. Once the patch is acceptable and assuming the patch applies cleanly to all versions to commit to, the core developer clicks a "Commit" button, fills in a commit message and NEWS entry, and everything gets committed (if the patch can't apply cleanly then the core developer does it the old-fashion way, or maybe auto-generate a new PR which can be manually touched up so it does apply cleanly?) Basically the ideal scenario lets contributors use whatever tools and platforms that they want and provides as much automated support as possible to make sure their code is tip-top before and during code review while core developers can review and commit patches so easily that they can do their job from a beach with a tablet and some WiFi. ## Where the current proposed solutions seem to fall short ### GitHub/Bitbucket Basically GitHub/Bitbucket is a win for contributors but doesn't buy core developers that much. GitHub/Bitbucket gives contributors the easy cloning, drive-by patches, CI, and PRs. Core developers get a code review tool -- I'm counting Rietveld as deprecated after Guido's comments about the code's maintenance issues -- and push-button commits **only for single branch changes**. But for any patch that crosses branches we don't really gain anything. At best core developers tell a contributor "please send your PR against 3.4", push-button merge it, update a local clone, merge from 3.4 to default, do the usual stuff, commit, and then push; that still keeps me off the beach, though, so that doesn't get us the whole way. You could force people to submit two PRs, but I don't see that flying. Maybe some tool could be written that automatically handles the merge/commit across branches once the initial PR is in? Or automatically create a PR that core developers can touch up as necessary and then accept that as well? Regardless, some solution is necessary to handle branch-crossing PRs. As for GitHub vs. Bitbucket, I personally don't care. I like GitHub's interface more, but that's personal taste. I like hg more than git, but that's also personal taste (and I consider a transition from hg to git a hassle but not a deal-breaker but also not a win). It is unfortunate, though, that under this scenario we would have to choose only one platform. It's also unfortunate both are closed-source, but that's not a deal-breaker, just a knock against if the decision is close. ### Our own infrastructure The shortcoming here is the need for developers, developers, developers! Everything outlined in the ideal scenario is totally doable on our own infrastructure with enough code and time (donated/paid-for infrastructure shouldn't be an issue). But historically that code and time has not materialized. Our code review tool is a fork that probably should be replaced as only Martin von Löwis can maintain it. Basically Ezio Melotti maintains the issue tracker's code. We don't exactly have a ton of people constantly going "I'm so bored because everything for Python's development infrastructure gets sorted so quickly!" A perfect example is that R. David Murray came up with a nice update for our workflow after PyCon but then ran out of time after mostly defining it and nothing ever became of it (maybe we can rectify that at PyCon?). Eric Snow has pointed out how he has written similar code for pulling PRs from I think GitHub to another code review tool, but that doesn't magically make it work in our infrastructure or get someone to write it and help maintain it (no offense, Eric). IOW our infrastructure can do anything, but it can't run on hopes and dreams. Commitments from many people to making this happen by a certain deadline will be needed so as to not allow it to drag on forever. People would also have to commit to continued maintenance to make this viable long-term. # Next steps I'm thinking first draft PEPs by February 1 to know who's all-in (8 weeks away), all details worked out in final PEPs and whatever is required to prove to me it will work by the PyCon language summit (4 months away). I make a decision by May 1, and then implementation aims to be done by the time 3.5.0 is cut so we can switch over shortly thereafter (9 months away). Sound like a reasonable timeline?

15 45

Cpython code and ...
by Ethan Furman 07 Jan '15

07 Jan '15

I found this: PyObject * PyBytes_FromFormat(const char *format, ...) { Can someone enlighten me on what the '...' means? -- ~Ethan~

2 2

python 2.7.9 regression in argparse?
by anatoly techtonik 06 Jan '15

06 Jan '15

https://github.com/nickstenning/honcho/pull/121 -- anatoly t.

5 4

Issue 22619 at bugs.python.org
by Dmitry Kazakov 06 Jan '15

06 Jan '15

Greetings. I'm sorry if I'm too insistent, but it's not truly rewarding to constantly improve a patch that no one appears to need. Again, I understand people are busy working and/or reviewing critical patches, but 2 months of inactivity is not right. Yes, I posted a message yesterday, but no one seemed to be bothered. In any case, I'll respect your decision about this patch and will never ask for a review of this patch again. Regards, Dmitry.

4 3

Help with finding tutors for Python, Linux, R, Perl, Octave, MATLAB and/or Cytoscape for yeast microarray analysis, next generation sequencing and constructing gene interaction networks
by thomas hahn 03 Jan '15

03 Jan '15

Help with finding tutors for Python, Linux, R, Perl, Octave, MATLAB and/or Cytoscape for yeast microarray analysis, next generation sequencing and constructing gene interaction networks Hi I am a visually impaired bioinformatics graduate student using microarray data for my master’s thesis aimed at deciphering the mechanism by which the yeast wild type can suppress the rise of free reactive oxygen species (ROS) induced by caloric restriction (CR) but the Atg15 and Erg6 knockout mutant cannot. Since my remaining vision is very limited I need very high magnification. But that makes my visual field very small. Therefore I need somebody to teach me how to use these programming environments, especially for microarray analysis, next generation sequencing and constructing gene and pathway interaction networks. This is very difficult for me to figure out without assistance because Zoomtext, my magnification and text to speech software, on which I am depending because I am almost blind, has problems reading out aloud many programming related websites to me. And even those websites it can read, it can only read sequentially from left to right and then from top to bottom. Unfortunately, this way of acquiring, finding, selecting and processing new information and answering questions is too tiresome, exhausting, ineffective and especially way too time consuming for graduating with a PhD in bioinformatics before my funding runs out despite being severely limited by my visual disability. I would also need help with writing a good literature review and applying the described techniques to my own yeast Affimetrix microarray dataset because I cannot see well enough to find all relevant publications on my own. Some examples for specific tasks I urgently need help with are: 1. Analyzing and comparing the three publically available microarray datasets that can be accessed at: A. http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE41860 B. http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE38635 C. http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE9217 2. Learning how to use the Affymetrics microarray analysis software for the Yeast 2 chip, which can be found at http://www.affymetrix.com/support/technical/libraryfilesmain.affx 3. For Cytoscape I need somebody, who can teach me how to execute the tutorials at the following links because due to my very limited vision field I cannot see tutorial and program interface simultaneously. A. http://opentutorials.cgl.ucsf.edu/index.php/Tutorial:Introduction_to_Cytosc… B. http://opentutorials.cgl.ucsf.edu/index.php/Tutorial:Filtering_and_Editing_… C. http://cytoscape.org/manual/Cytoscape2_8Manual.html#Import%20Fixed-Format%2… D. http://wiki.cytoscape.org/Cytoscape_User_Manual/Network_Formats 4. Learning how to use the TopGo R package to perform statistical analysis on GO enrichments. Since I am legally blind the rehab agency is giving me money to pay tutors for this purpose. Could you please help me getting in touch regarding this with anybody, who could potentially be interested in teaching me one on one thus saving me time for acquiring new information and skills, which I need to finish my thesis on time, so that I can remain eligible for funding to continue in my bioinformatics PhD program despite being almost blind? The tutoring can be done remotely via TeamViewer 5 and Skype. Hence, it does not matter where my tutors are physically located. Currently I have tutors in Croatia and UK. But since they both work full time jobs while working on their PhD dissertation they only have very limited time to teach me online. Could you therefore please forward this request for help to anybody, who could potentially be interested or, who could connect me to somebody, who might be, because my graduation and career depend on it? Who else would you recommend me to contact regarding this? Where else could I post this because I am in urgent need for help? Could you please contact me directly via email at Thomas.F.Hahn2(a)gmail.com and/or Skype at tfh002 because my text to speech software has problems to read out this website aloud to me? I thank you very much in advance for your thoughts, ideas, suggestions, recommendations, time, help, efforts and support. With very warm regards, *Thomas Hahn* 1) *Graduate student in the Joint Bioinformatics Program at the University of Arkansas at Little Rock (UALR) and the University of Arkansas Medical Sciences (UAMS) &* 2) *Research & Industry Advocate, Founder and Board Member of RADISH MEDICAL SOLUTIONS, INC. (**http://www.radishmedical.com/thomas-hahn/* <http://www.radishmedical.com/thomas-hahn/>*) * *Primary email: **Thomas.F.Hahn2(a)gmail.com* <Thomas.F.Hahn2(a)gmail.com> *Cell phone: 318 243 3940* *Office phone: 501 682 1440* *Office location: EIT 535* *Skype ID: tfh002* *Virtual Google Voice phone to reach me while logged into my email (i.e. * *Thomas.F.Hahn2(a)gmail.com* <Thomas.F.Hahn2(a)gmail.com>*), even when having no cell phone reception, e.g. in big massive buildings: *(501) 301-4890 <%28501%29%20301-4890> *Web links: * 1) https://ualr.academia.edu/ThomasHahn 2) https://www.linkedin.com/pub/thomas-hahn/42/b29/42 3) http://facebook.com/Thomas.F.Hahn <https://www.facebook.com/Thomas.F.Hahn> 4) https://twitter.com/Thomas_F_Hahn

2 1

More compact dictionaries with faster iteration
by Raymond Hettinger 03 Jan '15

03 Jan '15

The current memory layout for dictionaries is unnecessarily inefficient. It has a sparse table of 24-byte entries containing the hash value, key pointer, and value pointer. Instead, the 24-byte entries should be stored in a dense table referenced by a sparse table of indices. For example, the dictionary: d = {'timmy': 'red', 'barry': 'green', 'guido': 'blue'} is currently stored as: entries = [['--', '--', '--'], [-8522787127447073495, 'barry', 'green'], ['--', '--', '--'], ['--', '--', '--'], ['--', '--', '--'], [-9092791511155847987, 'timmy', 'red'], ['--', '--', '--'], [-6480567542315338377, 'guido', 'blue']] Instead, the data should be organized as follows: indices = [None, 1, None, None, None, 0, None, 2] entries = [[-9092791511155847987, 'timmy', 'red'], [-8522787127447073495, 'barry', 'green'], [-6480567542315338377, 'guido', 'blue']] Only the data layout needs to change. The hash table algorithms would stay the same. All of the current optimizations would be kept, including key-sharing dicts and custom lookup functions for string-only dicts. There is no change to the hash functions, the table search order, or collision statistics. The memory savings are significant (from 30% to 95% compression depending on the how full the table is). Small dicts (size 0, 1, or 2) get the most benefit. For a sparse table of size t with n entries, the sizes are: curr_size = 24 * t new_size = 24 * n + sizeof(index) * t In the above timmy/barry/guido example, the current size is 192 bytes (eight 24-byte entries) and the new size is 80 bytes (three 24-byte entries plus eight 1-byte indices). That gives 58% compression. Note, the sizeof(index) can be as small as a single byte for small dicts, two bytes for bigger dicts and up to sizeof(Py_ssize_t) for huge dict. In addition to space savings, the new memory layout makes iteration faster. Currently, keys(), values, and items() loop over the sparse table, skipping-over free slots in the hash table. Now, keys/values/items can loop directly over the dense table, using fewer memory accesses. Another benefit is that resizing is faster and touches fewer pieces of memory. Currently, every hash/key/value entry is moved or copied during a resize. In the new layout, only the indices are updated. For the most part, the hash/key/value entries never move (except for an occasional swap to fill a hole left by a deletion). With the reduced memory footprint, we can also expect better cache utilization. For those wanting to experiment with the design, there is a pure Python proof-of-concept here: http://code.activestate.com/recipes/578375 YMMV: Keep in mind that the above size statics assume a build with 64-bit Py_ssize_t and 64-bit pointers. The space savings percentages are a bit different on other builds. Also, note that in many applications, the size of the data dominates the size of the container (i.e. the weight of a bucket of water is mostly the water, not the bucket). Raymond

27 70