Mailman 3 April 2015 - Python-Dev

unittest test discovery and namespace packages
by Alex Shkop 17 Apr '15

17 Apr '15

Hello! There's an issue considering test discovery in unittest module. Basically it is about unittest module that doesn't find tests in namespace packages. For more info see issue http://bugs.python.org/issue23882. I'm willing to make a patch for this bug. But I need help to formulate how test discovery should work. Documentation states that all importable modules that match pattern will be loaded. This means that test modules inside namespace packages should be loaded too. But enabling this would change things drastically. For example now, running python -m unittest inside cpython source root does nothing. If we will enable test discovery inside namespace packages then this command will start running the whole python test suite in Lib/test/. So I'm looking for someone's help to clarify how test discovery should work. Thanks, Alex -- Issue in bugtracker - http://bugs.python.org/issue23882 Documentation for discover() method - https://docs.python.org/3.4/library/unittest.html#unittest.TestLoader.disco…

2 1

Do we need to sign Windows files with GnuPG?
by Larry Hastings 17 Apr '15

17 Apr '15

As of Python 3.5 Steve Dower has taken over the Windows builds of Python from Martin van Loewis. He's also taken over for 2.7--though Martin's still doing builds for 3.4. For both versions, Steve is using all-new tooling for the build process. The output is different, too; he's producing .exe installers instead of .msi installers, and he has snazzy new "web-based" installers where the initial download is small, then it downloads the rest dynamically. Steve's also changed the authentication process. His new installers rely on a Windows digital signature technology called Authenticode where the signature is built right into the .exe file. Windows platforms will automatically authenticate executables signed with Authenticode, so this is both secure and convenient. Martin's build process also digitally signed the files he built, but not using Authenticode (or at least I don't think so). Like the Mac and source code releases, his automation used GnuPG to produce separate ".asc" files containing digital signatures. This meant authentication was a manual process. The Authenticode approach sounds great. But there are advantages to the GnuPG approach too: * Using GnuPG means we can authenticate the files from any platform, not just Windows. If there were a security breach on the Python content delivery network, any developer could get GnuPG for their platform and authenticate that the installers are unmodified. If we use Authenitcode, * GnuPG is agnostic about the data it digitally signs. So, for example, Martin's build process digitally signs the Windows help file--the ".chm" file--produced by his build process. The help file Steve builds is currently completely unsigned; Steve says he can try signing it but he's not sure it'll work. Note that .chm files actually /can/ contain live code, so this is at least a plausible vector for attack. My Windows development days are firmly behind me. So I don't really have an opinion here. So I put it to you, Windows Python developers: do you care about GnuPG signatures on Windows-specific files? Or do you not care? //arry/ p.s. And, of course, my thanks to both Steve and Martin for their past and continuing service to the Python community! It's a pleasure working with each of them. (Both of them? I forget how English works.)

15 37

Not being able to compile: "make: *** [Programs/_freeze_importlib] Error 1"
by Facundo Batista 17 Apr '15

17 Apr '15

Hello! It's been a while since I compiled python and run the test suite. Now that I'm starting to work in a patch, wanted to do that as a sanity check, and even "./configure" finishing ok, "make" fails :/ (I'm on Ubuntu Trusty, all packages updated, all dependencies theorically installed). Full trace here: http://linkode.org/TgkzZw90JUaoodvYzU7zX6 Before going into a deep debug, I thought about sending a mail to see if anybode else hit this issue, if it's a common problem, if there's a known workaround. Thanks!! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

3 3

Python3 Stable ABI
by Zachary Ware 16 Apr '15

16 Apr '15

In issue23903, I've created a script that will produce PC/python3.def by scraping the header files in Include. There are are many many discrepencies between what my script generates and what is currently in the repository (diff below), but in every case I've checked the script has been right: what the script finds is actually exported as part of the limited API, but due to not being in the .def file it's not actually exported from python3.dll. Almost all of the differences are things that the script found that weren't present, but there are a couple things going the other way. The point of this message is to ask everybody who maintains anything in C to take a look through and make sure everything in their area is properly guarded (or not) by Py_LIMITED_API. Alternately, if somebody can find a bug in my script and brain that's finding too much stuff, that would be great too. Ideally, after this is all settled I'd like to add the script to both the Makefile and the Windows build system, such that PC/python3.def is always kept up to date and flags changes that weren't meant to be made. Regards, -- Zach (I'm afraid Gmail might mangle this beyond recognition, you can find the diff at http://bugs.python.org/review/23903/diff/14549/PC/python3.def if it does.) diff -r 24f2c0279120 PC/python3.def --- a/PC/python3.def Mon Apr 13 15:51:59 2015 -0500 +++ b/PC/python3.def Mon Apr 13 16:10:34 2015 -0500 @@ -1,13 +1,15 @@ ; This file specifies the import forwarding for python3.dll ; It is used when building python3dll.vcxproj +; Generated by python3defgen.py, DO NOT modify directly! LIBRARY "python3" EXPORTS + PyAST_FromNode=python35.PyAST_FromNode + PyAST_FromNodeObject=python35.PyAST_FromNodeObject + PyAST_Validate=python35.PyAST_Validate PyArg_Parse=python35.PyArg_Parse PyArg_ParseTuple=python35.PyArg_ParseTuple PyArg_ParseTupleAndKeywords=python35.PyArg_ParseTupleAndKeywords PyArg_UnpackTuple=python35.PyArg_UnpackTuple - PyArg_VaParse=python35.PyArg_VaParse - PyArg_VaParseTupleAndKeywords=python35.PyArg_VaParseTupleAndKeywords PyArg_ValidateKeywordArguments=python35.PyArg_ValidateKeywordArguments PyBaseObject_Type=python35.PyBaseObject_Type DATA PyBool_FromLong=python35.PyBool_FromLong @@ -39,7 +41,6 @@ PyCFunction_GetFlags=python35.PyCFunction_GetFlags PyCFunction_GetFunction=python35.PyCFunction_GetFunction PyCFunction_GetSelf=python35.PyCFunction_GetSelf - PyCFunction_New=python35.PyCFunction_New PyCFunction_NewEx=python35.PyCFunction_NewEx PyCFunction_Type=python35.PyCFunction_Type DATA PyCallIter_New=python35.PyCallIter_New @@ -58,6 +59,7 @@ PyCapsule_SetPointer=python35.PyCapsule_SetPointer PyCapsule_Type=python35.PyCapsule_Type DATA PyClassMethodDescr_Type=python35.PyClassMethodDescr_Type DATA + PyCmpWrapper_Type=python35.PyCmpWrapper_Type DATA PyCodec_BackslashReplaceErrors=python35.PyCodec_BackslashReplaceErrors PyCodec_Decode=python35.PyCodec_Decode PyCodec_Decoder=python35.PyCodec_Decoder @@ -68,6 +70,7 @@ PyCodec_IncrementalEncoder=python35.PyCodec_IncrementalEncoder PyCodec_KnownEncoding=python35.PyCodec_KnownEncoding PyCodec_LookupError=python35.PyCodec_LookupError + PyCodec_NameReplaceErrors=python35.PyCodec_NameReplaceErrors PyCodec_Register=python35.PyCodec_Register PyCodec_RegisterError=python35.PyCodec_RegisterError PyCodec_ReplaceErrors=python35.PyCodec_ReplaceErrors @@ -122,6 +125,7 @@ PyErr_Fetch=python35.PyErr_Fetch PyErr_Format=python35.PyErr_Format PyErr_FormatV=python35.PyErr_FormatV + PyErr_GetExcInfo=python35.PyErr_GetExcInfo PyErr_GivenExceptionMatches=python35.PyErr_GivenExceptionMatches PyErr_NewException=python35.PyErr_NewException PyErr_NewExceptionWithDoc=python35.PyErr_NewExceptionWithDoc @@ -132,14 +136,25 @@ PyErr_PrintEx=python35.PyErr_PrintEx PyErr_ProgramText=python35.PyErr_ProgramText PyErr_Restore=python35.PyErr_Restore + PyErr_SetExcFromWindowsErr=python35.PyErr_SetExcFromWindowsErr + PyErr_SetExcFromWindowsErrWithFilename=python35.PyErr_SetExcFromWindowsErrWithFilename + PyErr_SetExcFromWindowsErrWithFilenameObject=python35.PyErr_SetExcFromWindowsErrWithFilenameObject + PyErr_SetExcFromWindowsErrWithFilenameObjects=python35.PyErr_SetExcFromWindowsErrWithFilenameObjects + PyErr_SetExcInfo=python35.PyErr_SetExcInfo + PyErr_SetExcWithArgsKwargs=python35.PyErr_SetExcWithArgsKwargs PyErr_SetFromErrno=python35.PyErr_SetFromErrno PyErr_SetFromErrnoWithFilename=python35.PyErr_SetFromErrnoWithFilename PyErr_SetFromErrnoWithFilenameObject=python35.PyErr_SetFromErrnoWithFilenameObject + PyErr_SetFromErrnoWithFilenameObjects=python35.PyErr_SetFromErrnoWithFilenameObjects + PyErr_SetFromWindowsErr=python35.PyErr_SetFromWindowsErr + PyErr_SetFromWindowsErrWithFilename=python35.PyErr_SetFromWindowsErrWithFilename + PyErr_SetImportError=python35.PyErr_SetImportError PyErr_SetInterrupt=python35.PyErr_SetInterrupt PyErr_SetNone=python35.PyErr_SetNone PyErr_SetObject=python35.PyErr_SetObject PyErr_SetString=python35.PyErr_SetString PyErr_SyntaxLocation=python35.PyErr_SyntaxLocation + PyErr_SyntaxLocationEx=python35.PyErr_SyntaxLocationEx PyErr_WarnEx=python35.PyErr_WarnEx PyErr_WarnExplicit=python35.PyErr_WarnExplicit PyErr_WarnFormat=python35.PyErr_WarnFormat @@ -171,12 +186,21 @@ PyExc_AssertionError=python35.PyExc_AssertionError DATA PyExc_AttributeError=python35.PyExc_AttributeError DATA PyExc_BaseException=python35.PyExc_BaseException DATA + PyExc_BlockingIOError=python35.PyExc_BlockingIOError DATA + PyExc_BrokenPipeError=python35.PyExc_BrokenPipeError DATA PyExc_BufferError=python35.PyExc_BufferError DATA PyExc_BytesWarning=python35.PyExc_BytesWarning DATA + PyExc_ChildProcessError=python35.PyExc_ChildProcessError DATA + PyExc_ConnectionAbortedError=python35.PyExc_ConnectionAbortedError DATA + PyExc_ConnectionError=python35.PyExc_ConnectionError DATA + PyExc_ConnectionRefusedError=python35.PyExc_ConnectionRefusedError DATA + PyExc_ConnectionResetError=python35.PyExc_ConnectionResetError DATA PyExc_DeprecationWarning=python35.PyExc_DeprecationWarning DATA PyExc_EOFError=python35.PyExc_EOFError DATA PyExc_EnvironmentError=python35.PyExc_EnvironmentError DATA PyExc_Exception=python35.PyExc_Exception DATA + PyExc_FileExistsError=python35.PyExc_FileExistsError DATA + PyExc_FileNotFoundError=python35.PyExc_FileNotFoundError DATA PyExc_FloatingPointError=python35.PyExc_FloatingPointError DATA PyExc_FutureWarning=python35.PyExc_FutureWarning DATA PyExc_GeneratorExit=python35.PyExc_GeneratorExit DATA @@ -185,18 +209,23 @@ PyExc_ImportWarning=python35.PyExc_ImportWarning DATA PyExc_IndentationError=python35.PyExc_IndentationError DATA PyExc_IndexError=python35.PyExc_IndexError DATA + PyExc_InterruptedError=python35.PyExc_InterruptedError DATA + PyExc_IsADirectoryError=python35.PyExc_IsADirectoryError DATA PyExc_KeyError=python35.PyExc_KeyError DATA PyExc_KeyboardInterrupt=python35.PyExc_KeyboardInterrupt DATA PyExc_LookupError=python35.PyExc_LookupError DATA PyExc_MemoryError=python35.PyExc_MemoryError DATA - PyExc_MemoryErrorInst=python35.PyExc_MemoryErrorInst DATA PyExc_NameError=python35.PyExc_NameError DATA + PyExc_NotADirectoryError=python35.PyExc_NotADirectoryError DATA PyExc_NotImplementedError=python35.PyExc_NotImplementedError DATA PyExc_OSError=python35.PyExc_OSError DATA PyExc_OverflowError=python35.PyExc_OverflowError DATA PyExc_PendingDeprecationWarning=python35.PyExc_PendingDeprecationWarning DATA + PyExc_PermissionError=python35.PyExc_PermissionError DATA + PyExc_ProcessLookupError=python35.PyExc_ProcessLookupError DATA PyExc_RecursionErrorInst=python35.PyExc_RecursionErrorInst DATA PyExc_ReferenceError=python35.PyExc_ReferenceError DATA + PyExc_ResourceWarning=python35.PyExc_ResourceWarning DATA PyExc_RuntimeError=python35.PyExc_RuntimeError DATA PyExc_RuntimeWarning=python35.PyExc_RuntimeWarning DATA PyExc_StopIteration=python35.PyExc_StopIteration DATA @@ -205,6 +234,7 @@ PyExc_SystemError=python35.PyExc_SystemError DATA PyExc_SystemExit=python35.PyExc_SystemExit DATA PyExc_TabError=python35.PyExc_TabError DATA + PyExc_TimeoutError=python35.PyExc_TimeoutError DATA PyExc_TypeError=python35.PyExc_TypeError DATA PyExc_UnboundLocalError=python35.PyExc_UnboundLocalError DATA PyExc_UnicodeDecodeError=python35.PyExc_UnicodeDecodeError DATA @@ -215,6 +245,7 @@ PyExc_UserWarning=python35.PyExc_UserWarning DATA PyExc_ValueError=python35.PyExc_ValueError DATA PyExc_Warning=python35.PyExc_Warning DATA + PyExc_WindowsError=python35.PyExc_WindowsError DATA PyExc_ZeroDivisionError=python35.PyExc_ZeroDivisionError DATA PyException_GetCause=python35.PyException_GetCause PyException_GetContext=python35.PyException_GetContext @@ -242,10 +273,12 @@ PyGILState_Release=python35.PyGILState_Release PyGetSetDescr_Type=python35.PyGetSetDescr_Type DATA PyImport_AddModule=python35.PyImport_AddModule + PyImport_AddModuleObject=python35.PyImport_AddModuleObject PyImport_AppendInittab=python35.PyImport_AppendInittab PyImport_Cleanup=python35.PyImport_Cleanup PyImport_ExecCodeModule=python35.PyImport_ExecCodeModule PyImport_ExecCodeModuleEx=python35.PyImport_ExecCodeModuleEx + PyImport_ExecCodeModuleObject=python35.PyImport_ExecCodeModuleObject PyImport_ExecCodeModuleWithPathnames=python35.PyImport_ExecCodeModuleWithPathnames PyImport_GetImporter=python35.PyImport_GetImporter PyImport_GetMagicNumber=python35.PyImport_GetMagicNumber @@ -253,8 +286,10 @@ PyImport_GetModuleDict=python35.PyImport_GetModuleDict PyImport_Import=python35.PyImport_Import PyImport_ImportFrozenModule=python35.PyImport_ImportFrozenModule + PyImport_ImportFrozenModuleObject=python35.PyImport_ImportFrozenModuleObject PyImport_ImportModule=python35.PyImport_ImportModule PyImport_ImportModuleLevel=python35.PyImport_ImportModuleLevel + PyImport_ImportModuleLevelObject=python35.PyImport_ImportModuleLevelObject PyImport_ImportModuleNoBlock=python35.PyImport_ImportModuleNoBlock PyImport_ReloadModule=python35.PyImport_ReloadModule PyInterpreterState_Clear=python35.PyInterpreterState_Clear @@ -310,10 +345,18 @@ PyMapping_SetItemString=python35.PyMapping_SetItemString PyMapping_Size=python35.PyMapping_Size PyMapping_Values=python35.PyMapping_Values + PyMarshal_ReadObjectFromString=python35.PyMarshal_ReadObjectFromString + PyMarshal_WriteLongToFile=python35.PyMarshal_WriteLongToFile + PyMarshal_WriteObjectToFile=python35.PyMarshal_WriteObjectToFile + PyMarshal_WriteObjectToString=python35.PyMarshal_WriteObjectToString + PyMem_Calloc=python35.PyMem_Calloc PyMem_Free=python35.PyMem_Free PyMem_Malloc=python35.PyMem_Malloc PyMem_Realloc=python35.PyMem_Realloc PyMemberDescr_Type=python35.PyMemberDescr_Type DATA + PyMember_GetOne=python35.PyMember_GetOne + PyMember_SetOne=python35.PyMember_SetOne + PyMemoryView_FromMemory=python35.PyMemoryView_FromMemory PyMemoryView_FromObject=python35.PyMemoryView_FromObject PyMemoryView_GetContiguous=python35.PyMemoryView_GetContiguous PyMemoryView_Type=python35.PyMemoryView_Type DATA @@ -327,9 +370,15 @@ PyModule_GetFilename=python35.PyModule_GetFilename PyModule_GetFilenameObject=python35.PyModule_GetFilenameObject PyModule_GetName=python35.PyModule_GetName + PyModule_GetNameObject=python35.PyModule_GetNameObject PyModule_GetState=python35.PyModule_GetState PyModule_New=python35.PyModule_New + PyModule_NewObject=python35.PyModule_NewObject PyModule_Type=python35.PyModule_Type DATA + PyNode_AddChild=python35.PyNode_AddChild + PyNode_Free=python35.PyNode_Free + PyNode_ListTree=python35.PyNode_ListTree + PyNode_New=python35.PyNode_New PyNullImporter_Type=python35.PyNullImporter_Type DATA PyNumber_Absolute=python35.PyNumber_Absolute PyNumber_Add=python35.PyNumber_Add @@ -343,6 +392,7 @@ PyNumber_InPlaceAnd=python35.PyNumber_InPlaceAnd PyNumber_InPlaceFloorDivide=python35.PyNumber_InPlaceFloorDivide PyNumber_InPlaceLshift=python35.PyNumber_InPlaceLshift + PyNumber_InPlaceMatrixMultiply=python35.PyNumber_InPlaceMatrixMultiply PyNumber_InPlaceMultiply=python35.PyNumber_InPlaceMultiply PyNumber_InPlaceOr=python35.PyNumber_InPlaceOr PyNumber_InPlacePower=python35.PyNumber_InPlacePower @@ -355,6 +405,7 @@ PyNumber_Invert=python35.PyNumber_Invert PyNumber_Long=python35.PyNumber_Long PyNumber_Lshift=python35.PyNumber_Lshift + PyNumber_MatrixMultiply=python35.PyNumber_MatrixMultiply PyNumber_Multiply=python35.PyNumber_Multiply PyNumber_Negative=python35.PyNumber_Negative PyNumber_Or=python35.PyNumber_Or @@ -367,6 +418,7 @@ PyNumber_TrueDivide=python35.PyNumber_TrueDivide PyNumber_Xor=python35.PyNumber_Xor PyOS_AfterFork=python35.PyOS_AfterFork + PyOS_CheckStack=python35.PyOS_CheckStack PyOS_InitInterrupts=python35.PyOS_InitInterrupts PyOS_InputHook=python35.PyOS_InputHook DATA PyOS_InterruptOccurred=python35.PyOS_InterruptOccurred @@ -393,6 +445,7 @@ PyObject_CallMethod=python35.PyObject_CallMethod PyObject_CallMethodObjArgs=python35.PyObject_CallMethodObjArgs PyObject_CallObject=python35.PyObject_CallObject + PyObject_Calloc=python35.PyObject_Calloc PyObject_CheckReadBuffer=python35.PyObject_CheckReadBuffer PyObject_ClearWeakRefs=python35.PyObject_ClearWeakRefs PyObject_DelItem=python35.PyObject_DelItem @@ -405,6 +458,7 @@ PyObject_GC_UnTrack=python35.PyObject_GC_UnTrack PyObject_GenericGetAttr=python35.PyObject_GenericGetAttr PyObject_GenericSetAttr=python35.PyObject_GenericSetAttr + PyObject_GenericSetDict=python35.PyObject_GenericSetDict PyObject_GetAttr=python35.PyObject_GetAttr PyObject_GetAttrString=python35.PyObject_GetAttrString PyObject_GetItem=python35.PyObject_GetItem @@ -431,9 +485,10 @@ PyObject_SetItem=python35.PyObject_SetItem PyObject_Size=python35.PyObject_Size PyObject_Str=python35.PyObject_Str - PyObject_Type=python35.PyObject_Type DATA + PyObject_Type=python35.PyObject_Type PyParser_SimpleParseFileFlags=python35.PyParser_SimpleParseFileFlags PyParser_SimpleParseStringFlags=python35.PyParser_SimpleParseStringFlags + PyParser_SimpleParseStringFlagsFilename=python35.PyParser_SimpleParseStringFlagsFilename PyProperty_Type=python35.PyProperty_Type DATA PyRangeIter_Type=python35.PyRangeIter_Type DATA PyRange_Type=python35.PyRange_Type DATA @@ -474,8 +529,8 @@ PySlice_New=python35.PySlice_New PySlice_Type=python35.PySlice_Type DATA PySortWrapper_Type=python35.PySortWrapper_Type DATA + PyState_AddModule=python35.PyState_AddModule PyState_FindModule=python35.PyState_FindModule - PyState_AddModule=python35.PyState_AddModule PyState_RemoveModule=python35.PyState_RemoveModule PyStructSequence_GetItem=python35.PyStructSequence_GetItem PyStructSequence_New=python35.PyStructSequence_New @@ -484,9 +539,11 @@ PySuper_Type=python35.PySuper_Type DATA PySys_AddWarnOption=python35.PySys_AddWarnOption PySys_AddWarnOptionUnicode=python35.PySys_AddWarnOptionUnicode + PySys_AddXOption=python35.PySys_AddXOption PySys_FormatStderr=python35.PySys_FormatStderr PySys_FormatStdout=python35.PySys_FormatStdout PySys_GetObject=python35.PySys_GetObject + PySys_GetXOptions=python35.PySys_GetXOptions PySys_HasWarnOptions=python35.PySys_HasWarnOptions PySys_ResetWarnOptions=python35.PySys_ResetWarnOptions PySys_SetArgv=python35.PySys_SetArgv @@ -503,6 +560,24 @@ PyThreadState_New=python35.PyThreadState_New PyThreadState_SetAsyncExc=python35.PyThreadState_SetAsyncExc PyThreadState_Swap=python35.PyThreadState_Swap + PyThread_GetInfo=python35.PyThread_GetInfo + PyThread_ReInitTLS=python35.PyThread_ReInitTLS + PyThread_acquire_lock=python35.PyThread_acquire_lock + PyThread_acquire_lock_timed=python35.PyThread_acquire_lock_timed + PyThread_allocate_lock=python35.PyThread_allocate_lock + PyThread_create_key=python35.PyThread_create_key + PyThread_delete_key=python35.PyThread_delete_key + PyThread_delete_key_value=python35.PyThread_delete_key_value + PyThread_exit_thread=python35.PyThread_exit_thread + PyThread_free_lock=python35.PyThread_free_lock + PyThread_get_key_value=python35.PyThread_get_key_value + PyThread_get_stacksize=python35.PyThread_get_stacksize + PyThread_get_thread_ident=python35.PyThread_get_thread_ident + PyThread_init_thread=python35.PyThread_init_thread + PyThread_release_lock=python35.PyThread_release_lock + PyThread_set_key_value=python35.PyThread_set_key_value + PyThread_set_stacksize=python35.PyThread_set_stacksize + PyThread_start_new_thread=python35.PyThread_start_new_thread PyTraceBack_Here=python35.PyTraceBack_Here PyTraceBack_Print=python35.PyTraceBack_Print PyTraceBack_Type=python35.PyTraceBack_Type DATA @@ -561,34 +636,51 @@ PyUnicode_AsEncodedString=python35.PyUnicode_AsEncodedString PyUnicode_AsEncodedUnicode=python35.PyUnicode_AsEncodedUnicode PyUnicode_AsLatin1String=python35.PyUnicode_AsLatin1String + PyUnicode_AsMBCSString=python35.PyUnicode_AsMBCSString PyUnicode_AsRawUnicodeEscapeString=python35.PyUnicode_AsRawUnicodeEscapeString + PyUnicode_AsUCS4=python35.PyUnicode_AsUCS4 + PyUnicode_AsUCS4Copy=python35.PyUnicode_AsUCS4Copy PyUnicode_AsUTF16String=python35.PyUnicode_AsUTF16String PyUnicode_AsUTF32String=python35.PyUnicode_AsUTF32String PyUnicode_AsUTF8String=python35.PyUnicode_AsUTF8String PyUnicode_AsUnicodeEscapeString=python35.PyUnicode_AsUnicodeEscapeString PyUnicode_AsWideChar=python35.PyUnicode_AsWideChar - PyUnicode_ClearFreelist=python35.PyUnicode_ClearFreelist + PyUnicode_AsWideCharString=python35.PyUnicode_AsWideCharString + PyUnicode_BuildEncodingMap=python35.PyUnicode_BuildEncodingMap + PyUnicode_ClearFreeList=python35.PyUnicode_ClearFreeList PyUnicode_Compare=python35.PyUnicode_Compare + PyUnicode_CompareWithASCIIString=python35.PyUnicode_CompareWithASCIIString PyUnicode_Concat=python35.PyUnicode_Concat PyUnicode_Contains=python35.PyUnicode_Contains PyUnicode_Count=python35.PyUnicode_Count PyUnicode_Decode=python35.PyUnicode_Decode PyUnicode_DecodeASCII=python35.PyUnicode_DecodeASCII PyUnicode_DecodeCharmap=python35.PyUnicode_DecodeCharmap + PyUnicode_DecodeCodePageStateful=python35.PyUnicode_DecodeCodePageStateful PyUnicode_DecodeFSDefault=python35.PyUnicode_DecodeFSDefault PyUnicode_DecodeFSDefaultAndSize=python35.PyUnicode_DecodeFSDefaultAndSize PyUnicode_DecodeLatin1=python35.PyUnicode_DecodeLatin1 + PyUnicode_DecodeLocale=python35.PyUnicode_DecodeLocale + PyUnicode_DecodeLocaleAndSize=python35.PyUnicode_DecodeLocaleAndSize + PyUnicode_DecodeMBCS=python35.PyUnicode_DecodeMBCS + PyUnicode_DecodeMBCSStateful=python35.PyUnicode_DecodeMBCSStateful PyUnicode_DecodeRawUnicodeEscape=python35.PyUnicode_DecodeRawUnicodeEscape PyUnicode_DecodeUTF16=python35.PyUnicode_DecodeUTF16 PyUnicode_DecodeUTF16Stateful=python35.PyUnicode_DecodeUTF16Stateful PyUnicode_DecodeUTF32=python35.PyUnicode_DecodeUTF32 PyUnicode_DecodeUTF32Stateful=python35.PyUnicode_DecodeUTF32Stateful + PyUnicode_DecodeUTF7=python35.PyUnicode_DecodeUTF7 + PyUnicode_DecodeUTF7Stateful=python35.PyUnicode_DecodeUTF7Stateful PyUnicode_DecodeUTF8=python35.PyUnicode_DecodeUTF8 PyUnicode_DecodeUTF8Stateful=python35.PyUnicode_DecodeUTF8Stateful PyUnicode_DecodeUnicodeEscape=python35.PyUnicode_DecodeUnicodeEscape + PyUnicode_EncodeCodePage=python35.PyUnicode_EncodeCodePage + PyUnicode_EncodeFSDefault=python35.PyUnicode_EncodeFSDefault + PyUnicode_EncodeLocale=python35.PyUnicode_EncodeLocale PyUnicode_FSConverter=python35.PyUnicode_FSConverter PyUnicode_FSDecoder=python35.PyUnicode_FSDecoder PyUnicode_Find=python35.PyUnicode_Find + PyUnicode_FindChar=python35.PyUnicode_FindChar PyUnicode_Format=python35.PyUnicode_Format PyUnicode_FromEncodedObject=python35.PyUnicode_FromEncodedObject PyUnicode_FromFormat=python35.PyUnicode_FromFormat @@ -599,30 +691,28 @@ PyUnicode_FromStringAndSize=python35.PyUnicode_FromStringAndSize PyUnicode_FromWideChar=python35.PyUnicode_FromWideChar PyUnicode_GetDefaultEncoding=python35.PyUnicode_GetDefaultEncoding + PyUnicode_GetLength=python35.PyUnicode_GetLength PyUnicode_GetSize=python35.PyUnicode_GetSize + PyUnicode_InternFromString=python35.PyUnicode_InternFromString + PyUnicode_InternImmortal=python35.PyUnicode_InternImmortal + PyUnicode_InternInPlace=python35.PyUnicode_InternInPlace PyUnicode_IsIdentifier=python35.PyUnicode_IsIdentifier PyUnicode_Join=python35.PyUnicode_Join PyUnicode_Partition=python35.PyUnicode_Partition PyUnicode_RPartition=python35.PyUnicode_RPartition PyUnicode_RSplit=python35.PyUnicode_RSplit + PyUnicode_ReadChar=python35.PyUnicode_ReadChar PyUnicode_Replace=python35.PyUnicode_Replace PyUnicode_Resize=python35.PyUnicode_Resize PyUnicode_RichCompare=python35.PyUnicode_RichCompare - PyUnicode_SetDefaultEncoding=python35.PyUnicode_SetDefaultEncoding PyUnicode_Split=python35.PyUnicode_Split PyUnicode_Splitlines=python35.PyUnicode_Splitlines + PyUnicode_Substring=python35.PyUnicode_Substring PyUnicode_Tailmatch=python35.PyUnicode_Tailmatch PyUnicode_Translate=python35.PyUnicode_Translate - PyUnicode_BuildEncodingMap=python35.PyUnicode_BuildEncodingMap - PyUnicode_CompareWithASCIIString=python35.PyUnicode_CompareWithASCIIString - PyUnicode_DecodeUTF7=python35.PyUnicode_DecodeUTF7 - PyUnicode_DecodeUTF7Stateful=python35.PyUnicode_DecodeUTF7Stateful - PyUnicode_EncodeFSDefault=python35.PyUnicode_EncodeFSDefault - PyUnicode_InternFromString=python35.PyUnicode_InternFromString - PyUnicode_InternImmortal=python35.PyUnicode_InternImmortal - PyUnicode_InternInPlace=python35.PyUnicode_InternInPlace PyUnicode_Type=python35.PyUnicode_Type DATA - PyWeakref_GetObject=python35.PyWeakref_GetObject DATA + PyUnicode_WriteChar=python35.PyUnicode_WriteChar + PyWeakref_GetObject=python35.PyWeakref_GetObject PyWeakref_NewProxy=python35.PyWeakref_NewProxy PyWeakref_NewRef=python35.PyWeakref_NewRef PyWrapperDescr_Type=python35.PyWrapperDescr_Type DATA @@ -633,6 +723,8 @@ Py_BuildValue=python35.Py_BuildValue Py_CompileString=python35.Py_CompileString Py_DecRef=python35.Py_DecRef + Py_DecodeLocale=python35.Py_DecodeLocale + Py_EncodeLocale=python35.Py_EncodeLocale Py_EndInterpreter=python35.Py_EndInterpreter Py_Exit=python35.Py_Exit Py_FatalError=python35.Py_FatalError @@ -660,44 +752,95 @@ Py_NewInterpreter=python35.Py_NewInterpreter Py_ReprEnter=python35.Py_ReprEnter Py_ReprLeave=python35.Py_ReprLeave + Py_SetPath=python35.Py_SetPath Py_SetProgramName=python35.Py_SetProgramName Py_SetPythonHome=python35.Py_SetPythonHome Py_SetRecursionLimit=python35.Py_SetRecursionLimit Py_SymtableString=python35.Py_SymtableString Py_VaBuildValue=python35.Py_VaBuildValue + Py_hexdigits=python35.Py_hexdigits DATA + _PyDebug_PrintTotalRefs=python35._PyDebug_PrintTotalRefs + _PyDict_Dummy=python35._PyDict_Dummy + _PyDict_GetItemId=python35._PyDict_GetItemId + _PyDict_GetItemIdWithError=python35._PyDict_GetItemIdWithError + _PyDict_SetItemId=python35._PyDict_SetItemId _PyErr_BadInternalCall=python35._PyErr_BadInternalCall + _PyEval_FiniThreads=python35._PyEval_FiniThreads + _PyGILState_Reinit=python35._PyGILState_Reinit + _PyImportZip_Init=python35._PyImportZip_Init + _PyMethodWrapper_Type=python35._PyMethodWrapper_Type DATA + _PyNamespace_New=python35._PyNamespace_New + _PyNamespace_Type=python35._PyNamespace_Type DATA + _PyNone_Type=python35._PyNone_Type DATA + _PyNotImplemented_Type=python35._PyNotImplemented_Type DATA + _PyOS_GetOpt=python35._PyOS_GetOpt + _PyOS_IsMainThread=python35._PyOS_IsMainThread + _PyOS_SigintEvent=python35._PyOS_SigintEvent _PyObject_CallFunction_SizeT=python35._PyObject_CallFunction_SizeT + _PyObject_CallMethodId=python35._PyObject_CallMethodId + _PyObject_CallMethodIdObjArgs=python35._PyObject_CallMethodIdObjArgs + _PyObject_CallMethodId_SizeT=python35._PyObject_CallMethodId_SizeT _PyObject_CallMethod_SizeT=python35._PyObject_CallMethod_SizeT + _PyObject_GC_Calloc=python35._PyObject_GC_Calloc _PyObject_GC_Malloc=python35._PyObject_GC_Malloc _PyObject_GC_New=python35._PyObject_GC_New _PyObject_GC_NewVar=python35._PyObject_GC_NewVar _PyObject_GC_Resize=python35._PyObject_GC_Resize + _PyObject_GetAttrId=python35._PyObject_GetAttrId + _PyObject_HasAttrId=python35._PyObject_HasAttrId + _PyObject_IsAbstract=python35._PyObject_IsAbstract _PyObject_New=python35._PyObject_New _PyObject_NewVar=python35._PyObject_NewVar + _PyObject_SetAttrId=python35._PyObject_SetAttrId _PyState_AddModule=python35._PyState_AddModule + _PySys_SetObjectId=python35._PySys_SetObjectId + _PyThreadState_DeleteExcept=python35._PyThreadState_DeleteExcept _PyThreadState_Init=python35._PyThreadState_Init _PyThreadState_Prealloc=python35._PyThreadState_Prealloc _PyTrash_delete_later=python35._PyTrash_delete_later DATA _PyTrash_delete_nesting=python35._PyTrash_delete_nesting DATA _PyTrash_deposit_object=python35._PyTrash_deposit_object _PyTrash_destroy_chain=python35._PyTrash_destroy_chain + _PyTrash_thread_deposit_object=python35._PyTrash_thread_deposit_object + _PyTrash_thread_destroy_chain=python35._PyTrash_thread_destroy_chain + _PyUnicode_ClearStaticStrings=python35._PyUnicode_ClearStaticStrings + _PyUnicode_FromId=python35._PyUnicode_FromId _PyWeakref_CallableProxyType=python35._PyWeakref_CallableProxyType DATA _PyWeakref_ProxyType=python35._PyWeakref_ProxyType DATA _PyWeakref_RefType=python35._PyWeakref_RefType DATA + _Py_AddToAllObjects=python35._Py_AddToAllObjects _Py_BuildValue_SizeT=python35._Py_BuildValue_SizeT _Py_CheckRecursionLimit=python35._Py_CheckRecursionLimit DATA _Py_CheckRecursiveCall=python35._Py_CheckRecursiveCall _Py_Dealloc=python35._Py_Dealloc + _Py_DumpTraceback=python35._Py_DumpTraceback DATA + _Py_DumpTracebackThreads=python35._Py_DumpTracebackThreads DATA _Py_EllipsisObject=python35._Py_EllipsisObject DATA _Py_FalseStruct=python35._Py_FalseStruct DATA + _Py_ForgetReference=python35._Py_ForgetReference + _Py_GetAllocatedBlocks=python35._Py_GetAllocatedBlocks + _Py_GetRefTotal=python35._Py_GetRefTotal + _Py_HashSecret_Initialized=python35._Py_HashSecret_Initialized DATA + _Py_NegativeRefcount=python35._Py_NegativeRefcount + _Py_NewReference=python35._Py_NewReference _Py_NoneStruct=python35._Py_NoneStruct DATA _Py_NotImplementedStruct=python35._Py_NotImplementedStruct DATA + _Py_PrintReferenceAddresses=python35._Py_PrintReferenceAddresses + _Py_PrintReferences=python35._Py_PrintReferences + _Py_RefTotal=python35._Py_RefTotal DATA _Py_SwappedOp=python35._Py_SwappedOp DATA - _Py_TrueStruct=python35._Py_TrueStruct DATA _Py_VaBuildValue_SizeT=python35._Py_VaBuildValue_SizeT - _PyArg_Parse_SizeT=python35._PyArg_Parse_SizeT - _PyArg_ParseTuple_SizeT=python35._PyArg_ParseTuple_SizeT - _PyArg_ParseTupleAndKeywords_SizeT=python35._PyArg_ParseTupleAndKeywords_SizeT - _PyArg_VaParse_SizeT=python35._PyArg_VaParse_SizeT - _PyArg_VaParseTupleAndKeywords_SizeT=python35._PyArg_VaParseTupleAndKeywords_SizeT - _Py_BuildValue_SizeT=python35._Py_BuildValue_SizeT + _Py_add_one_to_index_C=python35._Py_add_one_to_index_C + _Py_add_one_to_index_F=python35._Py_add_one_to_index_F + _Py_device_encoding=python35._Py_device_encoding + _Py_fopen=python35._Py_fopen + _Py_fopen_obj=python35._Py_fopen_obj + _Py_read=python35._Py_read + _Py_stat=python35._Py_stat + _Py_wfopen=python35._Py_wfopen + _Py_wgetcwd=python35._Py_wgetcwd + _Py_wreadlink=python35._Py_wreadlink + _Py_wrealpath=python35._Py_wrealpath + _Py_write=python35._Py_write + _Py_write_noraise=python35._Py_write_noraise

3 2

Status on PEP-431 Timezones
by Lennart Regebro 15 Apr '15

15 Apr '15

Hi! I wrote PEP-431 two years ago, and never got around to implement it. This year I got some renewed motivation after Berker Peksağ made an effort of implementing it. I'm planning to work more on this during the PyCon sprints, and also have a BoF session or similar during the conference. Anyone interested in a session on this, mail me and we'll set up a time and place! //Lennart ------------------ If anyone is interested in the details of the problem, this is it. The big problem is the ambiguous times, like 02:30 a time when you move the clock back one hour, as there are two different 02:30's that day. I wrote down my experiences with looking into and trying to implement several different solutions. And the problem there is actually how to tell the datetime if it is before or after the changeover. == How others have solved it == === dateutil.tz: Ignore the problem === dateutil.tz simply ignores the problems with ambiguous datetimes, keeping them ambiguous. === pytz: One timezone instance per changeover === Pytz implements ambiguous datetimes by having one class per timezone. Each change in the UTC offset changes, either because of a DST changeover, or because the timezone changes, is represented as one instance of the class. All instances are held in a list which is a class attribute of the timezone class. You flag in which DST changeover you are by uising different instances as the datetimes tzinfo. Since the timezone this way knows if it is DST or not, the datetime as a whole knows if it's DST or not. Benefits: - Only known possible implementation without modifying stdlib, which of course was a requirement, as pytz is a third-party library. - DST offset can be quickly returned, as it does not need to be calculated. Drawbacks: - A complex and highly magical implementation of timezones that is hard to understand. - Required new normalize()/localize() functions on the timezone, and hence the API is not stdlib's API. - Hundreds of instances per timezone means slightly more memory usage. == Options for PEP 431 == === Stdlib option 0: Ignore it === I don't think this is an option, really. Listed for completness. === Stdlib option 1: One timezone instance per changeover === Option 1 is to do it like pytz, have one timezone instance per changeover. However, this is likely not possible to do without fundamentally changing the datetime API, or making it very hard to use. For example, when creating a datetime instance and passing in a tzinfo today this tzinfo is just attached to the datetime. But when having multiple instances of tzinfos this means you have to select the correct one to pass in. pytz solves this with the .localize() method, which let's the timezone class choose which instance to pass in. We can't pass in the timezone class into datetime(), because that would require datetime.__new__ to create new datetimes as a part of the timezone arithmetic. These in turn, would create new datetimes in __new__ as a part of the timezone arithmetic, which in turn, yeah, you get it... I haven't been able to solve that issue without either changing the API/usage, or getting infinite recursions. Benefits: - Proven soloution through pytz. - Fast dst() call. Drawbacks: - Trying to use this technique with the current API tends to create infinite recursions. It seems to require big API changes. - Slow datetime() instance creation. === Stdlib option 2: A datetime _is_dst flag === By having a flag on the datetime instance that says "this is in DST or not" the timezone implementation can be kept simpler. You also have to either calculate if the datetime is in a DST or not either when creating it, which demands datetime object creations, and causes infinite recursions, or you have to calculate it when needed, which means you can get "Ambiguous date time errors" at unexpected times later. Also, when trying to implement this, I get bogged down in the complexities of how tzinfo and datetime is calling each other back and forth, and when to pass in the current is_dst and when to pass in the the desired is_dst, etc. The API and current implementation is not designed with this case in mind, and it gets very tricky. Benefits: - Simpler tzinfo() implementations. Drawbacks: - It seems likely that we must change some API's. - This in turn may affect the pytz implementation. Or not, hard to say. - The DST offset must use slow timezone calculations. However, since datetimes are immutable it can be a cached, lazy, one-time operation. === Stdlib option 3: UTC internal representation === Having UTC as the internal representation makes the whole issue go away. Datetimes are no longer ambiguous, except when creating, so checks need to be done during creation, but that should be possible without datetime creation in this case, resolving the infinite recursion problem. Benefits: - Problem solved. - Minimal API changes. Drawbacks: - Backwards compatibility with pickles. - Possible other backwards incompatibility problems. - Both DST offset and date time display representation must use slow timezone calculations. However, since datetimes are immutable it can be a cached, lazy, one-time operation. I'm currently trying to implement solution #2 above. Feedback is welcome.

7 21

Re: [Python-Dev] Status on PEP-431 Timezones
by Isaac Schwabacher 15 Apr '15

15 Apr '15

On 15-04-15, Akira Li <4kir4.1i(a)gmail.com> wrote: > Isaac Schwabacher <ischwabacher(a)wisc.edu> writes: > > ... > > > > I know that you can do datetime.now(tz), and you can do datetime(2013, > > 11, 3, 1, 30, tzinfo=zoneinfo('America/Chicago')), but not being able > > to add a time zone to an existing naive datetime is painful (and > > strptime doesn't even let you pass in a time zone). > > `.now(tz)` is correct. `datetime(..., tzinfo=tz)`) is wrong: if tz is a > pytz timezone then you may get a wrong tzinfo (LMT), you should use > `tz.localize(naive_dt, is_dst=False|True|None)` instead. The whole point of this thread is to finalize PEP 431, which fixes the problem for which `localize()` and `normalize()` are workarounds. When this is done, `datetime(..., tzinfo=tz)` will be correct. ijs

2 1

Re: [Python-Dev] Status on PEP-431 Timezones
by Isaac Schwabacher 15 Apr '15

15 Apr '15

On 15-04-10, Stuart Bishop wrote: > > On 10 April 2015 at 17:12, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > > > The question of "store the DST flag" vs "store the offset" is essentially a > > data normalisation one - there's only a single bit of additional information > > actually needed (whether the time is DST or not in the annual hour of > > ambiguity), which can then be combined with the local time, the location and > > the zone info database to get the *actual* offset. One thing that hasn't been mentioned is that is_dst and offset are not parallel on the datetime.time class, since you have no information about when in the history of the time zone a time was recorded. If you only record the time zone and is_dst flag, then updating zoneinfo can change the UTC time corresponding to existing aware time objects. Whereas if you only store offsets, it can change whether a time is interpreted as being DST or not (or whether the offset is considered valid at all). Which is mostly to say that aware time objects that don't also store dates are probably just a bad idea in general. > The dst flag must be stored in the datetime, either as a boolean or > encoded in the timezone structure. If you only have the offset, you > lose interoperability with the time module (and the standard IANA > zoneinfo library, posix etc., which it wraps). (dt + > timedelta(hours=1)).ctime() will give an incorrect answer when you > cross a DST transition. >From local time + offset you can compute UTC time and from there you can lookup in the tzinfo whether it's DST or not. But yes, I'm starting to be less enamored of this idea. I get that it's basically the same as doing everything in UTC and caching local time, but I wasn't thinking about the fact that so many existing APIs need (local time, is_dst) that the friction is a problem. > > A question the PEP perhaps *should* consider is whether or not to offer an > > API allowing datetime objects to be built from a naive datetime, a fixed > > offset and a location, throwing NonExistentTimeError if the given date, time > > and offset doesn't match either the DST or non-DST times at that location. > > I don't think you need a specific API, apart from being able to > construct a tzinfo using nothing but an offset (lots of people require > this for things like parsing email headers, which is why pytz has the > FixedOffset class). > > datetime.now().astimezone(FixedOffset(-1200)).astimezone(timezone('Melbourne/Australia', > is_dst=None) This doesn't work: ``` >>> from datetime import * >>> datetime.now().astimezone(timezone(-timedelta(hours=2))) ValueError: astimezone() cannot be applied to a naive datetime ``` But also, how is this different from needing to know the offset in order to construct an aware datetime? I know that you can do datetime.now(tz), and you can do datetime(2013, 11, 3, 1, 30, tzinfo=zoneinfo('America/Chicago')), but not being able to add a time zone to an existing naive datetime is painful (and strptime doesn't even let you pass in a time zone). Some of the people who designed the instruments that we use to collect data understood why we would care when it was collected, and some of them didn't, and I need to be able to handle their data regardless. (The makers of one device with subsecond resolution opted for maximum compatibility with Microsoft Excel by writing a CSV with times as naive days since Dec. 30, 1899 to five decimal digits, but I doubt there's anything to be done for them.) In addition, it would be nice to able to say that going back and forth between aware and naive datetimes is evil and you should never do it, but it's a necessity if you want to be able to implement relative timedeltas in some form. Unless the stdlib wants to step up with a be-all, end-all implementation of "this time tomorrow" and friends, there shouldn't be unnecessary friction here. > > P.S. The description of NonExistentTimeError in the PEP doesn't seem quite > > right, as it currently says it will only be thrown if "is_dst=None", which > > seems like a copy and paste error from the definition of AmbiguousTimeError. > > The PEP is correct here. If you explicitly specify is_dst, you know > which side of the transition you are on and which offset to use. You > can calculate datetime(2000, 12, 31, 2, 0, 0, 0).astimezone(foo, > is_dst=False) using the non-DST offset and get an answer. It might not > have ever appeared on the clock on your wall, but it is better than a > punch in the face. If you want a punch in the face, is_dst=None > refuses to guess and you get the exception. Returning a DST time precisely when is_dst=False is passed isn't a punch in the face? I mean, I get what you're saying and I agree that "given that this time is skipped, which offset do I want to interpret it in?" is the right question to ask, but this is horribly counterintuitive to anyone who hasn't spent a lot of time pondering it. If nonexistent times are going to behave that way, the API needs to have a better solution to Naming Things. > (Except for those cases where the timezone offset is changed without a > DST transition, but that is rare enough everyone pretends they don't > exist) As far as method parameters go, you might as well just say the first time is DST and the second is STD and explain in the docs that "is_dst" is just a mnemonic. But if you have an is_dst flag on your datetimes, suddenly this is an issue. ijs

3 2

Re: [Python-Dev] Status on PEP-431 Timezones
by Isaac Schwabacher 15 Apr '15

15 Apr '15

On 15-04-08, Lennart Regebro wrote: > === Stdlib option 2: A datetime _is_dst flag === > > By having a flag on the datetime instance that says "this is in DST or not" > the timezone implementation can be kept simpler. Storing the offset itself instead of a flag makes things conceptually cleaner. You get a representation that's slightly harder to construct from the sorts of information you have lying around (tz name, naive datetime, is_dst flag) but is no harder to construct *and* validate, and then is easier to work with and harder to misuse. As an added bonus, you get a representation that's still meaningful when time changes happen for political rather than DST reasons. Pros: - tzinfo.utcoffset() and local->UTC conversions don't require zoneinfo access. - it's harder to represent "I know this time is DST but I don't know what tz it's in" [I put this in pro because I don't see how this kind of ambiguity can lead to anything but trouble, but ymmv] - this representation is meaningful regardless of whether a time zone has DST - this representation meaningfully disambiguates across time changes not resulting from DST Cons: - tzinfo.dst() requires zoneinfo access - tzinfo.tzname() requires zoneinfo access IF you want those horrible ambiguous abbreviations (is CST America/Chicago or America/Havana?) [I really wanted to put this in pro] - (datetime, offset, tz) triples require validation [but so do (datetime, is_dst, tz) triples, even though it's easy to pretend they don't] - constructing an aware datetime from a naive one, an is_dst flag, and a time zone requires zoneinfo access [but this is needed for the validation step anyway] On 15-04-08, Alexander Belopolsky wrote: > With datetime, we also have a problem that POSIX APIs don't have to deal with: local time > arithmetics. What is t + timedelta(1) when t falls on the day before DST change? How would > you set the isdst flag in the result? It's whatever time comes 60*60*24 seconds after t in the same time zone, because the timedelta class isn't expressive enough to represent anything but absolute time differences (nor should it be, IMO). But it might make sense to import dateutil.relativedelta or mxDateTime.RelativeDateTime into the stdlib to make relative time differences (same local time on the next day, for instance) possible. ijs

5 5

Re: [Python-Dev] Status on PEP-431 Timezones
by Isaac Schwabacher 15 Apr '15

15 Apr '15

On 15-04-15, Lennart Regebro wrote: > On Wed, Apr 15, 2015 at 12:40 PM, Isaac Schwabacher > <ischwabacher(a)wisc.edu> wrote: > > I am on the fence about EPOCH + offset as an internal representation. On the one hand, it is conceptually cleaner than the horrible byte-packing that exists today, but on the other hand, it has implications for future implementations of leap-second-awareness. For instance, offset measures the difference between the local time and UTC. But is it honest-to-goodness leap-second aware UTC, or is it really Unix time? This could be solved by giving each tzinfo a pointer to the UTC from which it is offset, but that sounds like a can of worms we don't want to open. But if don't and we store EPOCH + offset, we can't add leap second awareness to UTC without corrupting all persisted aware datetimes. > > > That's true, with separate values like there is now we can easily > allow 23:59:60 as a timestamp during leap seconds. I'm not entirely > sure it makes a big difference though, I don't think we ever wants to > deal with leap seconds by default. > > I don't think we ever want the standard arithmetic deal with leap > seconds anyway. > > datetime(2012, 6, 30, 23, 30) + timedelta(seconds=3600) returning > datetime(2012, 7, 1, 0, 29, 59) > > > I guess leap second implementations should rather have special > functions for arithmethics that deal with this. You need relative timedeltas to mitigate the pain of leap seconds, yes. But as soon as you have timedeltas that are capable of representing "this number of seconds into the next minute" ("one minute") as opposed to "sixty seconds", this isn't so much of a problem. Though of course subtraction will (and should!) continue to yield timedelta(seconds=3601). >From my perspective, the issue is more that the stdlib shouldn't rule out leap seconds. It's reasonable enough to expect users who actually want them to write appropriate tzinfo and timedelta classes, but we don't want to make that impossible the way the old tzinfo interface made DST-aware time zones impossible by insisting that implementers implement a function that wasn't mathematically a function. I need to think about this more before I can get a real rant going. ijs

1 0

Re: [Python-Dev] Status on PEP-431 Timezones
by Isaac Schwabacher 15 Apr '15

15 Apr '15

On 15-04-15, Lennart Regebro wrote: > On Wed, Apr 15, 2015 at 11:10 AM, Chris Angelico <rosuav(a)gmail.com> wrote: > > Bikeshed: Would arithmetic be based on UTC time or Unix time? It'd be > > more logical to describe it as "adding six hours means adding six > > hours to the UTC time", but it'd look extremely odd when there's a > > leap second. > > It would ignore leap seconds. If you want to call that unix time or > not is a matter of opinion. Hm. I guess the internal representation > *could* be EPOCH + offset, and local times could be calculated > properties, which could be cached (or possibly calculated at > creation). I am on the fence about EPOCH + offset as an internal representation. On the one hand, it is conceptually cleaner than the horrible byte-packing that exists today, but on the other hand, it has implications for future implementations of leap-second-awareness. For instance, offset measures the difference between the local time and UTC. But is it honest-to-goodness leap-second aware UTC, or is it really Unix time? This could be solved by giving each tzinfo a pointer to the UTC from which it is offset, but that sounds like a can of worms we don't want to open. But if don't and we store EPOCH + offset, we can't add leap second awareness to UTC without corrupting all persisted aware datetimes. Also, I didn't mention this before because I figured people were getting sick of my dumb idea, but another advantage of always caching the offset is that you can detect future datetimes that have been corrupted by zoneinfo changes. You need both absolute time and offset to be able to do this. ijs

2 1