Mailman 3 February 2012 - Python-Dev

Re: [Python-Dev] PEP 410 (Decimal timestamp): the implementation is ready for a review
by Victor Stinner Feb. 19, 2012

Feb. 19, 2012

> However, I am still -1 on the solution proposed by the PEP. I still think > that migrating to datetime use is a better way to go, rather than a > proliferation of the data types used to represent timestamps, along with an > API to specify the type of data returned. > > Let's look at each item in the PEPs rationale for discarding the use of > datetimes: Oh, I forgot to mention my main concern about datetime: many functions returning timestamp have an undefined starting point (an no timezone information ), and so cannot be converted to datetime: - time.clock(), time.wallclock(), time.monotonic(), time.clock_gettime() (except for CLOCK_REALTIME) - time.clock_getres() - signal.get/setitimer() - os.wait3(), os.wait4(), resource.getrusage() - etc. Allowing datetime.datetime type just for few functions (like datetime.datetime or time.time) but not the others (raise an exception) is not an acceptable solution. > I'm looking at a use case from my flufl.lock library: > > return datetime.datetime.fromtimestamp( > os.stat(self._lockfile).st_mtime) Keep your code but just add timestamp=decimal.Decimal argument to os.stat() to get high-resolution timestamps! (well, you would at least avoid loss of precision loss if datetime is not improved to support nanosecond.) > * datetime.datetime has ordering issues with daylight saving time (DST) in > the duplicate hour of switching from DST to normal time. > > Sure, but only for timezone-ful datetimes, right? I don't know enough this topic to answer. Martin von Loewis should answer to this question! > * datetime.datetime is not as well integrated than Epoch timestamps, some > functions don't accept this type as input. For example, os.utime() expects > a tuple of Epoch timestamps. > > So, by implication, Decimal is better integrated by virtue of its ability to > be coerced to floats and other numeric stack types? Yes. decimal.Decimal is already supported by all functions accepting float (all functions expecting timestamps). > Will users ever have to explicitly convert Decimal types to use other APIs? Sorry, I don't understand. What do you mean? > It bothers me that the PEP is proposing that users will now have to be > prepared to handle yet another (and potentially *many* more) data types coming > from what are essentially datetime-like APIs. Users only get decimal.Decimal if they ask explicitly for decimal.Decimal. By default, they will still get float. Most users don't care of nanoseconds :-) If a library choose to return Decimal instead of float, it's a change in the library API unrelated to the PEP. > If it really is impossible or suboptimal to build high resolution datetimes > and timedeltas, and to use them in these APIs, then at the very least, the PEP > needs a stronger rationale for why this is. IMO supporting nanosecond in datetime and timedelta is an orthogonal issue. And yes, the PEP should maybe give better arguments against datetime :-) I will update the PEP to mention the starting point issue. > In any case, thanks for your work in this (and so many other!) areas. You're welcome :)

18 69

problem after installing python 3.2.2
by Vahid Ghaderi Feb. 18, 2012

Feb. 18, 2012

hi i have downloaded and installed python 3.2.2 but still when i use python in terminal it show's: root@debian:~# python Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. how can i change the default to python 3.2.2? tHanks.

2 1

dll name for embedding?
by Egon Smiwa Feb. 18, 2012

Feb. 18, 2012

Hi all, I'm an app developer with a CPython dll in the folder of that app. In general, are there strict requirements about the dll name (a preference would be "python.dll" (easy to update (simple replace) ). I successfully used "python.dll" and a few standard modules, then I tried to use the sympy library and its import fails with an AV exception, unless I rename the dll back to the original "python32.dll" Is there an intrinsic filename requirement inside the CPython dll, modules, or are name-restrictions to be presumed only in case of third-party libs?

2 1

Summary of Python tracker Issues
by Python tracker Feb. 17, 2012

Feb. 17, 2012

ACTIVITY SUMMARY (2012-02-10 - 2012-02-17) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3257 (+11) closed 22567 (+44) total 25824 (+55) Open issues with patches: 1391 Issues opened (40) ================== #13609: Add "os.get_terminal_size()" function http://bugs.python.org/issue13609 reopened by Arfrever #13866: {urllib,urllib.parse}.urlencode should not use quote_plus http://bugs.python.org/issue13866 reopened by Stephen.Day #13989: gzip always returns byte strings, no text mode http://bugs.python.org/issue13989 opened by maubp #13990: Benchmarks: 2to3 failures on the py3 side http://bugs.python.org/issue13990 opened by francismb #13992: Segfault in PyTrash_destroy_chain http://bugs.python.org/issue13992 opened by Aaron.Staley #13997: Clearly explain the bare minimum Python 3 users should know ab http://bugs.python.org/issue13997 opened by ncoghlan #13998: Lookbehind assertions go behind the start position for the mat http://bugs.python.org/issue13998 opened by Devin Jeanpierre #13999: Queue references in multiprocessing doc points to Queue module http://bugs.python.org/issue13999 opened by sandro.tosi #14001: CVE-2012-0845 Python v2.7.2 / v3.2.2 (SimpleXMLRPCServer): DoS http://bugs.python.org/issue14001 opened by iankko #14002: distutils2 fails to install a package from PyPI on Python 2.7. http://bugs.python.org/issue14002 opened by pmoore #14003: __self__ on built-in functions is not as documented http://bugs.python.org/issue14003 opened by SpecLad #14004: Distutils filelist selects too many files on Windows http://bugs.python.org/issue14004 opened by jason.coombs #14005: IDLE Crash when running/saving a file http://bugs.python.org/issue14005 opened by Scott.Bowman #14006: Improve the documentation of xml.etree.ElementTree http://bugs.python.org/issue14006 opened by eli.bendersky #14007: xml.etree.ElementTree - XMLParser and TreeBuilder's doctype() http://bugs.python.org/issue14007 opened by eli.bendersky #14009: Clearer documentation for cElementTree http://bugs.python.org/issue14009 opened by eric.araujo #14010: deeply nested filter segfaults http://bugs.python.org/issue14010 opened by alex #14011: packaging should use shutil archiving functions transparently http://bugs.python.org/issue14011 opened by eric.araujo #14012: Misc tarfile fixes http://bugs.python.org/issue14012 opened by eric.araujo #14013: tarfile should expose supported formats http://bugs.python.org/issue14013 opened by eric.araujo #14014: codecs.StreamWriter.reset contract not fulfilled http://bugs.python.org/issue14014 opened by Jim.Jewett #14015: surrogateescape largely missing from documentation http://bugs.python.org/issue14015 opened by Jim.Jewett #14017: Make it easy to create a new TextIOWrapper based on an existin http://bugs.python.org/issue14017 opened by ncoghlan #14018: OS X installer does not detect bad symlinks created by Xcode 3 http://bugs.python.org/issue14018 opened by ned.deily #14019: Unify tests for str.format and string.Formatter http://bugs.python.org/issue14019 opened by ncoghlan #14020: Improve HTMLParser doc http://bugs.python.org/issue14020 opened by ezio.melotti #14023: bytes implied to be mutable http://bugs.python.org/issue14023 opened by SpecLad #14026: test_cmd_line_script should include more sys.argv checks http://bugs.python.org/issue14026 opened by ncoghlan #14027: distutils2 lack of pysetup.bat http://bugs.python.org/issue14027 opened by 勇刚.罗 #14030: Be more careful about selecting the compiler in distutils http://bugs.python.org/issue14030 opened by djc #14032: test_cmd_line_script prints undefined 'data' variable http://bugs.python.org/issue14032 opened by Jason.Yeo #14034: the example in argparse doc is too complex http://bugs.python.org/issue14034 opened by tshepang #14035: behavior of test.support.import_fresh_module http://bugs.python.org/issue14035 opened by flox #14036: urlparse insufficient port property validation http://bugs.python.org/issue14036 opened by zulla #14037: Allow grouping of argparse subparser commands in help output http://bugs.python.org/issue14037 opened by ncoghlan #14038: Packaging test support code raises exception http://bugs.python.org/issue14038 opened by vinay.sajip #14039: Add "metavar" argument to add_subparsers() in argparse http://bugs.python.org/issue14039 opened by ncoghlan #14040: Deprecate some of the module file formats http://bugs.python.org/issue14040 opened by pitrou #14042: json.dumps() documentation is slightly incorrect. http://bugs.python.org/issue14042 opened by tomchristie #14043: Speed-up importlib's _FileFinder http://bugs.python.org/issue14043 opened by pitrou Most recent 15 issues with no replies (15) ========================================== #14043: Speed-up importlib's _FileFinder http://bugs.python.org/issue14043 #14042: json.dumps() documentation is slightly incorrect. http://bugs.python.org/issue14042 #14039: Add "metavar" argument to add_subparsers() in argparse http://bugs.python.org/issue14039 #14038: Packaging test support code raises exception http://bugs.python.org/issue14038 #14032: test_cmd_line_script prints undefined 'data' variable http://bugs.python.org/issue14032 #14027: distutils2 lack of pysetup.bat http://bugs.python.org/issue14027 #14023: bytes implied to be mutable http://bugs.python.org/issue14023 #14019: Unify tests for str.format and string.Formatter http://bugs.python.org/issue14019 #14018: OS X installer does not detect bad symlinks created by Xcode 3 http://bugs.python.org/issue14018 #14015: surrogateescape largely missing from documentation http://bugs.python.org/issue14015 #14014: codecs.StreamWriter.reset contract not fulfilled http://bugs.python.org/issue14014 #14013: tarfile should expose supported formats http://bugs.python.org/issue14013 #14012: Misc tarfile fixes http://bugs.python.org/issue14012 #14011: packaging should use shutil archiving functions transparently http://bugs.python.org/issue14011 #13999: Queue references in multiprocessing doc points to Queue module http://bugs.python.org/issue13999 Most recent 15 issues waiting for review (15) ============================================= #14043: Speed-up importlib's _FileFinder http://bugs.python.org/issue14043 #14040: Deprecate some of the module file formats http://bugs.python.org/issue14040 #14036: urlparse insufficient port property validation http://bugs.python.org/issue14036 #14035: behavior of test.support.import_fresh_module http://bugs.python.org/issue14035 #14020: Improve HTMLParser doc http://bugs.python.org/issue14020 #14013: tarfile should expose supported formats http://bugs.python.org/issue14013 #14012: Misc tarfile fixes http://bugs.python.org/issue14012 #14009: Clearer documentation for cElementTree http://bugs.python.org/issue14009 #14001: CVE-2012-0845 Python v2.7.2 / v3.2.2 (SimpleXMLRPCServer): DoS http://bugs.python.org/issue14001 #13974: packaging: test for set_platform() http://bugs.python.org/issue13974 #13973: urllib.parse is imported twice in xmlrpc.client http://bugs.python.org/issue13973 #13970: frameobject should not have f_yieldfrom attribute http://bugs.python.org/issue13970 #13969: path name must always be string (or None) http://bugs.python.org/issue13969 #13968: Support recursive globs http://bugs.python.org/issue13968 #13967: also test for an empty pathname http://bugs.python.org/issue13967 Top 10 most discussed issues (10) ================================= #13992: Segfault in PyTrash_destroy_chain http://bugs.python.org/issue13992 15 msgs #13609: Add "os.get_terminal_size()" function http://bugs.python.org/issue13609 14 msgs #13997: Clearly explain the bare minimum Python 3 users should know ab http://bugs.python.org/issue13997 14 msgs #13703: Hash collision security issue http://bugs.python.org/issue13703 11 msgs #14004: Distutils filelist selects too many files on Windows http://bugs.python.org/issue14004 8 msgs #14001: CVE-2012-0845 Python v2.7.2 / v3.2.2 (SimpleXMLRPCServer): DoS http://bugs.python.org/issue14001 7 msgs #14036: urlparse insufficient port property validation http://bugs.python.org/issue14036 7 msgs #13198: Remove duplicate definition of write_record_file http://bugs.python.org/issue13198 6 msgs #13579: string.Formatter doesn't understand the !a conversion specifie http://bugs.python.org/issue13579 6 msgs #13882: PEP 410: Use decimal.Decimal type for timestamps http://bugs.python.org/issue13882 6 msgs Issues closed (44) ================== #7644: bug in nntplib.body() method with possible fix http://bugs.python.org/issue7644 closed by pitrou #9750: sqlite3 iterdump fails on column with reserved name http://bugs.python.org/issue9750 closed by python-dev #10227: Improve performance of MemoryView slicing http://bugs.python.org/issue10227 closed by skrah #10287: NNTP authentication should check capabilities http://bugs.python.org/issue10287 closed by pitrou #11836: multiprocessing.queues.SimpleQueue is undocumented http://bugs.python.org/issue11836 closed by sandro.tosi #12297: Clarifications to atexit.register and unregister doc http://bugs.python.org/issue12297 closed by eric.araujo #13014: _ssl.c: refleak http://bugs.python.org/issue13014 closed by pitrou #13015: _collectionsmodule.c: refleak http://bugs.python.org/issue13015 closed by pitrou #13020: structseq.c: refleak http://bugs.python.org/issue13020 closed by pitrou #13089: parsetok.c: memory leak http://bugs.python.org/issue13089 closed by skrah #13092: pep-393: memory leaks #2 http://bugs.python.org/issue13092 closed by pitrou #13619: Add a new codec: "locale", the current locale encoding http://bugs.python.org/issue13619 closed by haypo #13878: test_sched failures on Windows buildbot http://bugs.python.org/issue13878 closed by neologix #13913: utf-8 or utf8 or utf-8 (codec display name inconsistency) http://bugs.python.org/issue13913 closed by haypo #13930: lib2to3 ability to output files into a different directory and http://bugs.python.org/issue13930 closed by gregory.p.smith #13948: rm needless use of set function http://bugs.python.org/issue13948 closed by eric.araujo #13949: rm needless use of pass statement http://bugs.python.org/issue13949 closed by eric.araujo #13950: rm commented-out code http://bugs.python.org/issue13950 closed by eric.araujo #13960: Handling of broken comments in HTMLParser http://bugs.python.org/issue13960 closed by ezio.melotti #13961: Have importlib use os.replace() http://bugs.python.org/issue13961 closed by brett.cannon #13972: set and frozenset constructors don't accept multiple iterables http://bugs.python.org/issue13972 closed by petri.lehtinen #13977: importlib simplification http://bugs.python.org/issue13977 closed by brett.cannon #13979: Automatic *libc.so loading behaviour http://bugs.python.org/issue13979 closed by meador.inge #13987: Handling of broken markup in HTMLParser on 2.7 http://bugs.python.org/issue13987 closed by ezio.melotti #13988: Expose the C implementation of ElementTree by default when imp http://bugs.python.org/issue13988 closed by flox #13991: namespace packages depending on order http://bugs.python.org/issue13991 closed by eric.araujo #13993: Handling of broken end tags in HTMLParser http://bugs.python.org/issue13993 closed by ezio.melotti #13994: incomplete revert in 2.7 Distutils left two copies of customiz http://bugs.python.org/issue13994 closed by ned.deily #13995: sqlite3 Cursor.rowcount documentation for old sqlite bug http://bugs.python.org/issue13995 closed by python-dev #13996: "What's New in Python" should have initial release date on hea http://bugs.python.org/issue13996 closed by rhettinger #14000: Subprocess stdin.flush does not flush http://bugs.python.org/issue14000 closed by rosslagerwall #14008: Python uses the new source when reporting an old exception http://bugs.python.org/issue14008 closed by flox #14016: Usage of socket.sendall() in multiple threads http://bugs.python.org/issue14016 closed by r.david.murray #14021: Write pkg_info with local encoding(GBK) will be a problem. http://bugs.python.org/issue14021 closed by eric.araujo #14022: bug in pkgutil.py with suggested fix http://bugs.python.org/issue14022 closed by ned.deily #14024: logging.Formatter Cache Prevents Exception Format Overriding http://bugs.python.org/issue14024 closed by vinay.sajip #14025: unittest.TestCase.assertEqual does not show diff when comparin http://bugs.python.org/issue14025 closed by michael.foord #14028: random.choice hits ValueError: cannot convert float NaN to int http://bugs.python.org/issue14028 closed by gregory.p.smith #14029: When using setattr identifiers can start with any character http://bugs.python.org/issue14029 closed by loewis #14031: logging module cannot format str.format log messages http://bugs.python.org/issue14031 closed by vinay.sajip #14033: distutils problem with setup.py build &setup.py install vs dir http://bugs.python.org/issue14033 closed by 勇刚.罗 #14041: bsddb DB_RUNRECOVERY crash on write access http://bugs.python.org/issue14041 closed by jcea #1326113: Letting "build_ext --libraries" take more than one lib http://bugs.python.org/issue1326113 closed by eric.araujo #1051216: make distutils.core.run_setup re-entrant http://bugs.python.org/issue1051216 closed by eric.araujo

1 0

A new dictionary implementation
by Mark Shannon Feb. 17, 2012

Feb. 17, 2012

Hi, Now that issue 13703 has been largely settled, I want to propose my new dictionary implementation again. It is a little more polished than before. https://bitbucket.org/markshannon/hotpy_new_dict Object-oriented benchmarks use considerably less memory and are sometimes faster (by a small amount). (I've only benchmarked on my old 32bit machine) E.g 2to3 No speed change -28% memory GCbench +10% speed -47% memory Other benchmarks show little or no change in behaviour, mainly minor memory savings. If an application is OO and uses lots of memory the new dict will save a lot of memory and maybe boost performance. Other applications will be largely unaffected. It passes all the tests. (I had to change a couple that relied on dict repr() ordering) Cheers, Mark.

14 35

Re: [Python-Dev] [Python-checkins] cpython: Disabling a test that fails on some bots. Will investigate the failure soon
by Nick Coghlan Feb. 17, 2012

Feb. 17, 2012

On Fri, Feb 17, 2012 at 2:09 AM, eli.bendersky <python-checkins(a)python.org> wrote: > diff --git a/Lib/test/test_xml_etree_c.py b/Lib/test/test_xml_etree_c.py > --- a/Lib/test/test_xml_etree_c.py > +++ b/Lib/test/test_xml_etree_c.py > @@ -53,8 +53,8 @@ > # actual class. In the Python version it's a class. > self.assertNotIsInstance(cET.Element, type) > > - def test_correct_import_cET_alias(self): > - self.assertNotIsInstance(cET_alias.Element, type) > + #def test_correct_import_cET_alias(self): > + #self.assertNotIsInstance(cET_alias.Element, type) While this one was fixed quickly, *please* don't comment tests out without some kind of explanation in the code (not just in the checkin message). Even better is to use the expected_failure() decorator or the skip() decorator. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 1

Counting collisions for the win
by Victor Stinner Feb. 16, 2012

Feb. 16, 2012

Hi, I'm working on the hash collision issue since 2 or 3 weeks. I evaluated all solutions and I think that I have now a good knowledge of the problem and how it should be solved. The major issue is to have a minor or no impact on applications (don't break backward compatibility). I saw three major solutions: - use a randomized hash - use two hashes, a randomized hash and the actual hash kept for backward compatibility - count collisions on dictionary lookup Using a randomized hash does break a lot of tests (e.g. tests relying on the representation of a dictionary). The patch is huge, too big to backport it directly on stable versions. Using a randomized hash may also break (indirectly) real applications because the application output is also somehow "randomized". For example, in the Django test suite, the HTML output is different at each run. Web browsers may render the web page differently, or crash, or ... I don't think that Django would like to sort attributes of each HTML tag, just because we wanted to fix a vulnerability. Randomized hash has also a major issue: if the attacker is able to compute the secret, (s)he can easily compute collisions and exploit the hash collision vulnerability again. I don't know exactly how complex it is to compute the secret, but our hash function is weak (it is far from being cryptographic, it is really simple to run it backward). If someone writes a fast function to compute the secret, we will go back to the same point. IMO using two hashes has the same disavantages of the randomized hash solution, whereas it is more complex to implement. The last solution is very simple: count collision and raise an exception if it hits a limit. The path is something like 10 lines whereas the randomized hash is more close to 500 lines, add a new file, change Visual Studio project file, etc. First I thaught that it would break more applications than the randomized hash, but I tried on Django: the test suite fails with a limit of 20 collisions, but not with a limit of 50 collisions, whereas the patch uses a limit of 1000 collisions. According to my basic tests, a limit of 35 collisions requires a dictionary with more than 10,000,000 integer keys to raise an error. I am not talking about the attack, but valid data. More details about my tests on the Django test suite: http://bugs.python.org/issue13703#msg151620 -- I propose to solve the hash collision vulnerability by counting collisions because it does fix the vulnerability with a minor or no impact on applications or backward compatibility. I don't see why we should use a different fix for Python 3.3. If counting collisons solves the issue for stable versions, it is also enough for Python 3.3. We now know all issues of the randomized hash solution, and I think that there are more drawbacks than advantages. IMO the randomized hash is overkill to fix the hash collision issue. I just have some requests on Marc Andre Lemburg patch: - the limit should be configurable: a new function in the sys module should be enough. It may be private (or replaced by an environment variable?) in stable versions - the set type should also be patched (I didn't check if it is vulnerable or not using the patch) - the patch has no test! (a class with a fixed hash should be enough to write a test) - the limit must be documented somwhere - the exception type should be different than KeyError Victor

31 76

plugging the hash attack
by Benjamin Peterson Feb. 16, 2012

Feb. 16, 2012

Hello everyone, In effort to get a fix out before Perl 6 goes mainstream, Barry and I have decided to pronounce on what we want for our stable releases. What we have decided is that 1. Simple hash randomization is the way to go. We think this has the best chance of actually fixing the problem while being fairly straightforward such that we're comfortable putting it in a stable release. 2. It will be off by default in stable releases and enabled by an envar at runtime. This will prevent code breakage from dictionary order changing as well as people depending on the hash stability. -- Regards, Benjamin

10 10

Store timestamps as decimal.Decimal objects
by Victor Stinner Feb. 16, 2012

Feb. 16, 2012

Hi, In issues #13882 and #11457, I propose to add an argument to functions returning timestamps to choose the timestamp format. Python uses float in most cases whereas float is not enough to store a timestamp with a resolution of 1 nanosecond. I added recently time.clock_gettime() to Python 3.3 which has a resolution of a nanosecond. The (first?) new timestamp format will be decimal.Decimal because it is able to store any timestamp in any resolution without loosing bits. Instead of adding a boolean argument, I would prefer to support more formats. My last patch provides the following formats: - "float": float (used by default) - "decimal": decimal.Decimal - "datetime": datetime.datetime - "timespec": (sec, nsec) tuple # I don't think that we need it, it is just another example The proposed API is: time.time(format="datetime") time.clock_gettime(time.CLOCK_REALTIME, format="decimal") os.stat(path, timestamp="datetime) etc. This API has an issue: importing the datetime or decimal object is implicit, I don't know if it is really an issue. (In my last patch, the import is done too late, but it can be fixed, it is not really a matter.) Alexander Belopolsky proposed to use time.time(format=datetime.datetime) instead. -- The first step would be to add an argument to functions returning timestamps. The second step is to accept these new formats (Decimal?) as input, for datetime.datetime.fromtimestamp() and os.utime() for example. (Using decimal.Decimal, we may remove os.utimens() and use the right function depending on the timestamp resolution.) -- I prefer Decimal over a dummy tuple like (sec, nsec) because you can do arithmetic on it: t2-t1, a+b, t/k, etc. It stores also the resolution of the clock: time.time() and time.clock_gettime() have for example different resolution (sec, ms, us for time.time() and ns for clock_gettime()). The decimal module is still implemented in Python, but there is working implementation in C which is much faster. Store timestamps as Decimal can be a motivation to integrate the C implementation :-) -- Examples with the time module: $ ./python Python 3.3.0a0 (default:52f68c95e025+, Jan 26 2012, 21:54:31) >>> import time >>> time.time() 1327611705.948446 >>> time.time('decimal') Decimal('1327611708.988419') >>> t1=time.time('decimal'); t2=time.time('decimal'); t2-t1 Decimal('0.000550') >>> t1=time.time('float'); t2=time.time('float'); t2-t1 5.9604644775390625e-06 >>> time.clock_gettime(time.CLOCK_MONOTONIC, 'decimal') Decimal('1211833.389740312') >>> time.clock_getres(time.CLOCK_MONOTONIC, 'decimal') Decimal('1E-9') >>> time.clock() 0.12 >>> time.clock('decimal') Decimal('0.120000') Examples with os.stat: $ ./python Python 3.3.0a0 (default:2914ce82bf89+, Jan 30 2012, 23:07:24) >>> import os >>> s=os.stat("setup.py", timestamp="datetime") >>> s.st_mtime - s.st_ctime datetime.timedelta(0) >>> print(s.st_atime - s.st_ctime) 52 days, 1:44:06.191293 >>> os.stat("setup.py", timestamp="timespec").st_ctime (1323458640, 702327236) >>> os.stat("setup.py", timestamp="decimal").st_ctime Decimal('1323458640.702327236') Victor

11 28

best place for an atomic file API
by Charles-François Natali Feb. 16, 2012

Feb. 16, 2012

Hi, Issue #8604 aims at adding an atomic file API to make it easier to create/update files atomically, using rename() on POSIX systems and MoveFileEx() on Windows (which are now available through os.replace()). It would also use fsync() on POSIX to make sure data is committed to disk. For example, it could be used by importlib to avoid races when writting bytecode files (issues #13392, #13003, #13146), or more generally by any application that wants to make sure to end up with a consistent file even in face of crash (e.g. it seems that mercurial implemented their own version). Basically the usage would be, e.g.: with AtomicFile('foo') as f: pickle.dump(obj, f) or with AtomicFile('foo') as f: chunk = heavyCrunch() f.write(chunk) chunk = CrunchSomeMore() f.write(chunk) What would be the best place for a such a class? _pyio, tempfile, or a new atomicfile Cheers, cf

9 11