[Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

[Gregory P. Smith]

On Tue, Mar 19, 2019, 4:53 AM Ned Batchelder <ned at nedbatchelder.com> wrote:

> On 3/19/19 4:13 AM, Serhiy Storchaka wrote:
> > 19.03.19 00:41, Raymond Hettinger пише:
> >> 4) Fix the tests in the third-party modules to be more focused on
> >> their actual test objectives, the semantics of the generated XML
> >> rather than the exact serialization.  This option would seem like the
> >> right-thing-to-do but it isn't trivial because the entire premise of
> >> the existing test is invalid.  For every case, we'll actually have to
> >> think through what the test objective really is.
> >
>
> Option 4 is misleading.  Is anyone here really offering to "fix the
> tests in third-party modules"?  Option 4 is actually, "do nothing, and
> let a multitude of projects figure out how to fix their tests, slowing
> progress in those projects as they try to support Python 3.8."
>

We've done Option 4 for every past behavior change of any form on feature
.Next releases.  We do try to encourage projects to run their tests on the
3.Next betas so that they can be ready before 3.Next.0 lands, some of us
even do it ourselves when we're interested.  Many things won't get ready
ahead of time, but the actual .0 release forces the issue as their users
start demanding it on occasion offering patches.  We don't bock a release
on existing user code being ready for it.

In my case, the test code has a generic function to compare an actual
> directory of files to an expected directory of files, so it isn't quite
> as simple as "just use the right XML comparison."  And I support Python
> 2.7, 3.5, etc, so tests still need to work under those versions.  None
> of this is impossible, but please try not to preach to us maintainers
> that we are doing it wrong, that it will be easy to fix, etc.  Using
> language like "the entire premise of the test is invalid" seems
> needlessly condescending.
>

Agreed, that was poor wording.  Lets not let that type of wording escape
python-dev into docs about a behavior change.

Wording aside, a test relying on undefined behavior is testing for things
the code under test doesn't actually need to care about being true, even if
it has happened to work for years.  Such a test is overspecified.
Potentially without the authors previously consciously realizing that.
It'll need refactoring to loosen its requirements.  How to loosen it is
always an implementation detail based on the constraints imposed upon the
test.  Difficulty lies within range(0, "Port Mercurial to Python 3").  But
the end result is nice: The code is healthier as tests focus more on what
was actually important rather than what was quicker to write that got the
original job done many years ago.

One of the suggested solutions, a DOM comparison is not enough. I
> don't just want to know that my actual XML is different than my expected
> XML.  I want to know where and how it differs.
>
> Textual comparison may be the "wrong" way to check XML, but it gives me
> many tools for working with the test results.  It was simple and it
> worked.  Now in Python 3.8, because Python doesn't want to add an
> optional flag to continue doing what it has always done, I need to
> re-engineer my tests.
>
> --Ned.
>

I understand that from a code owners perspective having to do any work, no
matter what the reason, is counted as re-engineering.  But that doesn't
make it wrong.  If "what it has always done" was unspecified and arbitrary
despite happening to not change in the past rather than something easy to
continue the stability of such as sorted attributes, it is a burden to
maintain that **unspecifiable** behavior in the language or library going
forward.

(Note that I have no idea what order the xml code in question happened to
impose upon attributes; if it went from sorted to not a "fix" to provide
users is clear: provide a way to keep it sorted for those who need that.
If it relied on insertion order or hash table iteration order or the phase
of the moon when the release was cut, we are right to refuse to maintain
unspecifiable implementation side effect behavior)

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190319/396956a6/attachment.html>