xs:NCName puzzler

[GaryGapinski]

I have found that NCNames containing leading or trailing whitespace provoke no validation errors but cannot reliably be used in XPath references. The same problem is encountered even if these were constrained to xs:ID or xs:IDREF. xs:NCName, xs:ID, and xsd:IDREF all have "whiteSpace = collapse" facets in their definitions

E.g.,

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="https://github.com/raw/usnistgov/OSCAL/master/xml/schema/oscal_catalog_schema.xsd" schematypens="http://www.w3.org/2001/XMLSchema" title="OSCAL Catalog schema"?>
<catalog xmlns="http://csrc.nist.gov/ns/oscal/1.0" uuid="fafe5776-b69f-41ea-a5cf-2192bdc4242b">
    <metadata>
        <title>catalog 1</title>
        <last-modified>2021-04-22T17:09:41Z</last-modified>
        <version>one</version>
        <oscal-version>1.0.0-rc1</oscal-version>
    </metadata>
    <control id="   control-1   ">
        <title>Control one</title>
        <param id="param-1">
            <label>organization-defined time</label>
        </param>
        <part name="statement" id="statement-1">
            <p>Open the perimeter gate <insert type="param" id-ref="param-1"/>:</p>
        </part>
    </control>
</catalog>

has three spaces surrounding the control id attribute (<control id="␢␢␢control-1␢␢␢">), validates correctly, but the following

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:math="http://www.w3.org/2005/xpath-functions/math" exclude-result-prefixes="xs math" version="3.0"
    xmlns:oscal="http://csrc.nist.gov/ns/oscal/1.0" xpath-default-namespace="http://csrc.nist.gov/ns/oscal/1.0">
    <xsl:output method="text"/>
    <xsl:template match="/">
        <xsl:text expand-text="true">//control/@id contains «{//control/@id}»&#x0a;</xsl:text>
        <xsl:text expand-text="true">"if (//control[@id = 'control-1']) then 'true' else 'false'" is {if (//control[@id = 'control-1']) then 'true' else 'false'} &#x0a;</xsl:text>
        <xsl:text expand-text="true">"if (//control[@id = ' control-1 ']) then 'true' else 'false'" is {if (//control[@id = ' control-1 ']) then 'true' else 'false'} &#x0a;</xsl:text>
        <xsl:text expand-text="true">"if (//control[@id = '   control-1   ']) then 'true' else 'false'" is {if (//control[@id = '   control-1   ']) then 'true' else 'false'} &#x0a;</xsl:text>
    </xsl:template>
</xsl:stylesheet>

produces

//control/@id contains «   control-1   »
"if (//control[@id = 'control-1']) then 'true' else 'false'" is false 
"if (//control[@id = ' control-1 ']) then 'true' else 'false'" is false 
"if (//control[@id = '   control-1   ']) then 'true' else 'false'" is true 

Note that the whitespace has been processed as if "whiteSpace preserve" were in effect. "whiteSpace collapse" should consider leading and trailing whitespace as inconsequential

The above was tested with XML Editor 23.1 with the transformation using Saxon-PE 9.9.1.7. Validation also succeeds using xmllint (xmllint: using libxml version 20910).

I have not yet tried a variant schema with an extension of xs:NCName constrained to exclude whitespace.

[wendellpiez]

I suggest we replace or supplement NCName with a datatype along the lines of tokenOnly (appropriate string of characters, no whitespace). This would inhibit the introduction of "padded values" to begin with, thus forestalling the question of normalization.

Not relying on NCName at all (simple extending string) would also address the root of the problem since it would turn whitespace normalization off (under schema-aware parsing) so that " control1" and "control1" are not the same.

We can also broaden the scope of this effort and consider whether at this point we should not scrub our datatypes of problematic DTD-based data formats altogether. (They survive from early versions of OSCAL when their convenience outweighed their cost. But we have the infrastructure now.) Related to @GaryGapinski's original observation, the wiggliness around these data types also impacts XML-JSON conversion -- effectively it permits silent bugs on the XML side.

Questions for @david-waltermire-nist:

• what do you think of the tokenOnly datatype idea?
• how broadly should we scope this effort? timing is not bad - is it limited to NCName?

[wendellpiez]

This should also address usnistgov/metaschema#67 and usnistgov/metaschema#68 thanks @GaryGapinski very timely indeed.

[david-waltermire]

Changes to address this were integrated in PR #948. These changes will be released with OSCAL 1.0.0.