[ expat-Bugs-569461 ] 1.95.3 and new OASIS xml test suite

Sun Jul 7 08:40:02 2002

Bugs item #569461, was opened at 2002-06-15 15:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=569461&group_id=10127

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Rolf Ade (pointsman)
Assigned to: Karl Waclawek (kwaclaw)
Summary: 1.95.3 and new OASIS xml test suite

Initial Comment:
I've tested expat-1.95.3 (with xmltok.c updated to rev.
1.17, becase of bug 566240, all other files are the
original 1.95.3) against the recently updated OASIS xml
test suite (XML 1.0 (Second Edition) errata 20020320,
W3C Conformance Test Suite 20020606), avaliable via
http://www.w3.org/XML/Test/ and found a few new
problems, that are not triggered by older versions of
this test suite.

As in previous reports, I checked all
not-wellformedness tests (should
all raise error) and all valid tests (should all pass)
of the
test-suites xmltest, ibm, sun and oasis with xmlwf -p.
Especially for the well-formedness tests, I have _not_
throughout checked if the error reason, reported by
expat is the expected error, but checked only
mechanical, if the test has raised an error, regardless
of the exact error reason. This method is clearly not
perfect, and this time we have an example, that
underlines this.

ibm/not-wf/P32/ibm32n09.xml

This is a new test, not included in previous versions.
Problem is, that the standalone document declaration
has the value "yes" and there is an external markup
declaration of an entity (other than amp, lt, gt, apos,
quot). xmlwf -p doesn't report an error. The not
well-formedness problem is, that standalone="yes"
means, that all informations needed to build the XML
infoset must be found in the document entity
(standalone="yes" doesn't mean, that the document must
not have an external subset or external PE's, only that
this external entities doesn't change - per attribute
defaults or as in this case, entity declarations -
change the info in the document entity. See the last
sentence of "Well-Formedness Constraint: Entity
Declared" (P68).

ibm/not-wf/P68/ibm68n06.xml

Same reason as the test befor. This test _was_ present
in previous versions of the test suite. But with the
previous version of the external subset of this test,
xmlwf claimed a "syntax error" error in the external
subset, which I plain can't understand (eventually an
other expat bug?), but is clearly not the expected
error. In the new version of the test suite, this
external subset now has an XML declaration with
explicite encoding (the older version had only an XML
declaration without encoding) and is accepted by expat.

xmltest/not-wf/not-sa/010.xml
xmltest/not-wf/not-sa/011.xml

This tests are new in this edition of the test suite.
Unfortunately, this both tests seems to be not
documented, either in the test files isself nor in the
documentation file xmlconf-20020606.htm. As far as I
see, this tests test "Validity Constraint: Proper
Declaration/PE Nesting" (P29).

xmltest/not-wf/not-sa/005.xml

This test raised error with previous expat versions,
but does not anymore due to the changes, discussed in
bug 548690. This is intentional, according to the
548690 discussion. This test is now listed under "XML
Documents with Optional Errors". The test suite
documentation says:

"Conforming XML 1.0 Processors are permitted to ignore
certain errors, or to report them at user option. In
this section of this test report are found descriptions
of test cases which fit into this category.

Processor behavior on such test cases does not affect
conformance to the XML 1.0 (Second Edition)
Recommendation, except as noted." 

So, according to this, it's OK, that expat doesn't
report an error for this case. Since both reporting
error and not reporting error are OK, it may be
debatably, which behavior is more convenient for the
expat user. (Karl: ;-))

sun/not-wf/not-sa03.xml

This is a new test in this edition of the test suite.
Unfortunately, this test seems not to be documented. As
far as I see, it tests the same as
xmltest/not-wf/not-sa/005.xml

Tests, that still are wrong, as in previous versions are

ibm/not-wf/misc/432gewf.xml
sun/not-wf/uri01.xml

These are already discussed in the past.

Well, that's all.

rolf

----------------------------------------------------------------------

>Comment By: Karl Waclawek (kwaclaw)
Date: 2002-07-07 11:39

Message:
Logged In: YES 
user_id=290026

Patch #587161 should fix some of the problems,
but I specifically made no attempt to fix the problems
Expat has with:
- xmltest/not-wf/not-sa/010.xml and
- xmltest/not-wf/not-sa/011.xml.

Reason: It turns out, after consulting with the mailing
list for the XML test suite, public-xml-testsuite@w3.org,
that these two violate WFC: PE Between Declarations.
There is no quick and easy fix for this in Expat,
and I would have to spend some time thinking about it,
which I don't have at the moment.

Karl

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2002-06-17 23:15

Message:
Logged In: YES 
user_id=290026

Assigned to me, but only for the three test cases
described in my last message.

Karl

----------------------------------------------------------------------

Comment By: Rolf Ade (pointsman)
Date: 2002-06-17 19:21

Message:
Logged In: YES 
user_id=13222

Agreed

----------------------------------------------------------------------

Comment By: Karl Waclawek (kwaclaw)
Date: 2002-06-17 15:05

Message:
Logged In: YES 
user_id=290026

Given an improved understanding of section 4.1
in the XML spec, I will try to fix the following
test cases in the next Expat release:

ibm/not-wf/P32/ibm32n09.xml,
ibm/not-wf/P68/ibm68n06.xml and
sun/not-wf/not-sa03.xml 

In my opinion, the third one is not the same type as
xmltest/not-wf/not-sa/005.xml, but the same type
as the other two.

About the test cases
xmltest/not-wf/not-sa/010.xml and
xmltest/not-wf/not-sa/011.xml:

If they really check validity constraint P29, as Rolf
has suggested, then it is OK that Expat does not report
an error.

So, If I am successful, we would be left with only:
ibm/not-wf/misc/432gewf.xml and
sun/not-wf/uri01.xml,
conformance with which does not seem a 100% 
necessity, as previously discussed.

Karl

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110127&aid=569461&group_id=10127