Updating schemas to current model (transitional); updated docs in res…

…ulting XSD per usnistgov#448; some updates to markdown documentation

build/metaschema/lib/metaschema.xsd

@@ -108,6 +108,7 @@
     </xs:annotation>
     <xs:complexType>
       <xs:sequence>
+        <xs:element minOccurs="0" ref="m:formal-name"/>
         <xs:element minOccurs="0" ref="m:description"/>
         <xs:element minOccurs="0" ref="m:remarks"/>
       </xs:sequence>

build/metaschema/metaschema-notes.md

@@ -1,6 +1,6 @@
 # Metaschema Design and Terminology
 
-OSCAL Metaschema is a descriptive format for the specification and support of OSCAL tagging. Each metaschema is used as the basis for producing schema files, conversion files, documentation and utilities in support of that format. For any given OSCAL format defined by a metaschema (such as **catalog** or **profile**), the XML Schema (XSD) and JSON Schema will be consistent because they are produced from the same source.
+OSCAL Metaschema is a descriptive format for the specification and support of OSCAL tagging. A metaschema is used as the basis for producing schema files, conversion files, documentation and utilities in support of that format. For any given OSCAL format defined by a metaschema (such as **catalog** or **profile**), the XML Schema (XSD) and JSON Schema will be consistent because they are produced from the same source.
 
 However, the terminology used to describe data in either format will be different depending on whether XML or JSON is used. (As a notation for an object-oriented model, YAML will be similar to JSON.) While both data formats describe tree structures (directed graphs), each format (with its implicit data model) has its peculiar design, which requires specification in detail. A data point represented as an attribute on an element in XML, for example, might be a string property on a data object in JSON. The metaschema moderates this distinction by providing rules regarding its own semantic constructs and how they are to be represented in the target notation. Accordingly, a mapping between JSON and XML conceptual descriptions for any OSCAL format is possible, given the metaschema.
 
@@ -10,16 +10,14 @@ Within OSCAL models, all constructs are optional unless marked otherwise.
 
 | OSCAL Metaschema | XML | JSON \| YAML |
 |------------------|-----|------|
-| Assembly | A single element with element content (child of assembly element parent) |  Object property (of parent assembly object) |
-| Assemblies | A sequence of elements (element content) *each named as an individual* | Array property of parent object, with a key for the group, containing objects |
+| Assembly | An element with element content |  An object, either a property or a member of an array property |
 | Field (with no flags) | A single element with text content | String property |
-| Field with one or more flags | An element with text content, flags as attributes |   Object property with `STRVALUE` String property, flags as other properties |
-| Field `as='mixed'`, no flags permitted | An element permitting mixed content inline | String property, parseable as markdown |
-| Field `as='mixed'`, flag(s) permitted | idem | Object property with `RICHTEXT` String property parseable as markdown |
-| Fields | A sequence elements each for a single field (as above), named singly | Array property, with a key *for the group*, containing objects or strings as above (for fields with and without flags) |
+| Field with one or more flags | An element with text content, flags as attributes |   An object property with a designated property for its nominal string value as well as properties for its flags |
 | Flag | Attribute | String property |
-| Flag with type | Attribute with lexical constraints per type | String property with lexical constraints per type, or typed property such as `number` or URI (per type) | 
-| Prose | Sequence of markdown-convertible elements (`p`, `ul` etc.) | Array property with key=`prose` containing (markdown) strings |
+| Flag with designated data type | Attribute with lexical constraints per type | String property with lexical constraints per type, or typed property such as `number` or URI (per type) | 
+| Field `as-type='simple-markup'`, no flags permitted | An element permitting mixed content inline | String property or map with string property, parseable as markdown (line only) |
+| Field `as='complex-markup'`, flag(s) permitted | idem | Object property with `RICHTEXT` String property or object with string property, parseable as markdown (full blocks) |
+
 
 ## Constraints imposed on the Metaschema
 

build/metaschema/xml/produce-xsd.xsl

@@ -82,28 +82,9 @@
 
     <xsl:template match="namespace"/>
 
-    <xsl:template match="/METASCHEMA/schema-name">
-        <xsl:comment>
-            <xsl:apply-templates/>
-        </xsl:comment>
-        <xsl:text>&#xA;</xsl:text>
-    </xsl:template>
-
-    <xsl:template match="/METASCHEMA/short-name">
-        <xsl:comment>short name: <xsl:apply-templates/></xsl:comment>
-        <xsl:text>&#xA;</xsl:text>
-    </xsl:template>
-
-    <xsl:template match="/METASCHEMA/schema-version">
-        <xsl:comment>oscal-version: <xsl:apply-templates/></xsl:comment>
-        <xsl:text>&#xA;</xsl:text>
-    </xsl:template>
-
-    <xsl:template match="/METASCHEMA/remarks/*">
-        <xsl:comment>
-            <xsl:apply-templates/>
-        </xsl:comment>
-        <xsl:text>&#xA;</xsl:text>
+    <xsl:template match="/METASCHEMA/schema-name | /METASCHEMA/short-name |
+        /METASCHEMA/schema-version | /METASCHEMA/remarks/*">
+        <xsl:copy-of select="."/>
     </xsl:template>
 
     <xsl:template match="field | assembly">
@@ -159,6 +140,9 @@
 
     <xsl:template match="define-flag | define-field | define-assembly | flag[exists(formal-name| description)]" mode="annotated">
         <xs:annotation>
+            <xs:appInfo>
+                <xsl:copy-of select="formal-name, description"/>
+            </xs:appInfo>
             <xs:documentation>
                 <xsl:apply-templates select="formal-name, description"/>
             </xs:documentation>
@@ -252,23 +236,36 @@
             <xsl:apply-templates mode="#current"/>
     </xsl:template>
 
-<!-- dropping top level (placeholder) 'prose' element declaration and its complexType -->
+    <!-- dropping top level (placeholder) 'prose' element declaration and its complexType -->
     <xsl:template match="xs:schema/xs:element[@name='prose'] | xs:schema/xs:complexType[@name='prose']" mode="acquire-prose"/>
 
+    <!-- Mode 'wire-ns' wires up namespaces, bringing the prose module into the same namespace as the main
+     schema, and cleaning up declarations and names as it goes. -->
     <xsl:template match="comment()" mode="wire-ns">
         <xsl:copy-of select="."/>
     </xsl:template>
 
     <xsl:template match="*" mode="wire-ns">
         <xsl:copy copy-namespaces="no"> 
+            <xsl:namespace name="m">http://csrc.nist.gov/ns/oscal/metaschema/1.0</xsl:namespace>
             <xsl:namespace name="{$declaration-prefix}" select="$target-namespace"/>
             <xsl:namespace name="oscal-prose" select="$target-namespace"/>
             <xsl:copy-of select="@*"/>
             <xsl:apply-templates mode="#current"/>
         </xsl:copy>
     </xsl:template>
 
-    <xsl:template match="xs:documentation//text()" mode="wire-ns">
+    <xsl:template match="m:*" mode="wire-ns" xmlns:m="http://csrc.nist.gov/ns/oscal/metaschema/1.0">
+        <xsl:element name="m:{local-name()}">
+            <xsl:namespace name="m">http://csrc.nist.gov/ns/oscal/metaschema/1.0</xsl:namespace>
+            <xsl:namespace name="{$declaration-prefix}" select="$target-namespace"/>
+            <xsl:namespace name="oscal-prose" select="$target-namespace"/>
+            <xsl:copy-of select="@*"/>
+            <xsl:apply-templates mode="#current"/>
+        </xsl:element>
+    </xsl:template>
+
+    <xsl:template match="xs:documentation//text() | m:*//text()" mode="wire-ns">
         <xsl:copy-of select="."/>
     </xsl:template>
 

src/metaschema/oscal_catalog_metaschema.xml

@@ -24,14 +24,14 @@
   <define-assembly name="catalog" group-as="control-catalog">
       <formal-name>Catalog</formal-name>
       <description>A collection of controls.</description>
-      <flag datatype="ID" required="yes" ref="id"/>
+      <flag ref="id" datatype="ID" required="yes"/>
       <model>
-         <assembly ref="metadata" required="yes"/>
+         <assembly ref="metadata" min-occurs="1"/>
          <!--<field named="declarations"/>-->
          <!--<assemblies named="section"/>-->
          <choice>
-            <assemblies ref="group"/>
-            <assemblies ref="control"/>
+            <assembly ref="group" max-occurs="unbounded"/>
+            <assembly ref="control" max-occurs="unbounded"/>
          </choice>
          <assembly ref="back-matter">
             <description>Back matter including references and resources.</description>
@@ -88,16 +88,16 @@
   <define-assembly name="group" group-as="groups">
       <formal-name>Control Group</formal-name>
       <description>A group of controls, or of groups of controls.</description>
-      <flag datatype="ID" ref="id"/>
+      <flag ref="id" datatype="ID"/>
       <flag ref="class"/>
       <model>
-         <field ref="title" required="yes"/>
-         <assemblies ref="param"/>
-         <fields ref="prop"/>
-         <assemblies ref="part"/>
+         <field ref="title" min-occurs="1"/>
+         <assembly ref="param" max-occurs="unbounded"/>
+         <field ref="prop" max-occurs="unbounded"/>
+         <assembly ref="part" max-occurs="unbounded"/>
          <choice>
-            <assemblies ref="group"/>
-            <assemblies ref="control"/>
+            <assembly ref="group" max-occurs="unbounded"/>
+            <assembly ref="control" max-occurs="unbounded"/>
          </choice>
          <!--<assembly named="ref-list"/>-->
          <any/>
@@ -120,16 +120,16 @@
   <define-assembly name="control" group-as="controls">
       <formal-name>Control</formal-name>
       <description>A structured information object representing a security or privacy control. Each security or privacy control within the Catalog is defined by a distinct control instance.</description>
-      <flag datatype="ID" required="yes" ref="id"/>
+      <flag ref="id" datatype="ID" required="yes"/>
       <flag ref="class"/>
       <model>
-         <field ref="title" required="yes"/>
-         <assemblies ref="param"/>
+         <field ref="title" min-occurs="1"/>
+         <assembly ref="param" max-occurs="unbounded"/>
          <!-- TODO: Need to be able to add valid values in context -->
-         <fields ref="prop"/>
-         <fields ref="link"/>
-         <assemblies ref="part"/>
-         <assemblies ref="subcontrol"/>
+         <field ref="prop" max-occurs="unbounded"/>
+         <field ref="link" max-occurs="unbounded"/>
+         <assembly ref="part" max-occurs="unbounded"/>
+         <assembly ref="subcontrol" max-occurs="unbounded"/>
          <!--<assembly named="ref-list"/>-->
          <any/>
       </model>
@@ -147,16 +147,14 @@
   <define-assembly name="subcontrol" group-as="subcontrols">
       <formal-name>Sub-Control</formal-name>
       <description>A control extension or enhancement</description>
-      <flag datatype="ID" required="yes" ref="id"/>
+      <flag ref="id" datatype="ID" required="yes"/>
       <flag ref="class"/>
       <model>
-         <field ref="title" required="yes">
-            <description>The subcontrol's human-readable name.</description>
-         </field>
-         <assemblies ref="param"/>
-         <fields ref="prop"/>
-         <fields ref="link"/>
-         <assemblies ref="part"/>
+         <field ref="title" min-occurs="1"/>
+         <assembly ref="param" max-occurs="unbounded"/>
+         <field ref="prop" max-occurs="unbounded"/>
+         <field ref="link" max-occurs="unbounded"/>
+         <assembly ref="part" max-occurs="unbounded"/>
          <!--<assembly named="ref-list"/>-->
          <any/>
       </model>
@@ -170,31 +168,19 @@
   <define-assembly name="param" group-as="parameters">
       <formal-name>Parameter</formal-name>
       <description>Parameters provide a mechanism for the dynamic assignment of value(s) in a control.</description>
-      <flag datatype="ID" required="yes" ref="id"/>
+      <flag ref="id" datatype="ID" required="yes"/>
       <flag ref="class"/>
-      <flag datatype="IDREF" ref="depends-on"/>
+      <flag ref="depends-on" datatype="IDREF"/>
       <model>
          <field ref="label">
             <description>A short name for the parameter.</description>
             <remarks>
                <p>The label value should be suitable for inline display in a rendered catalog.</p>
             </remarks>
          </field>
-         <fields ref="usage">
-            <description>A short summary of the parameter's intended use.</description>
-            <remarks>
-               <p> A label is optional, but should be provided unless a <code>select</code> (selection) is provided.</p>
-            </remarks>
-         </fields>
-         <fields ref="constraint">
-            <description>A rule describing the permissible parameter values.</description>
-            <remarks>
-               <p>Currently, OSCAL does not standardize any formal rules language for value constraints. A <code>test</code> option may be used to specify a formal rule that may be automatically used if recognized by an OSCAL tool. Further development is <a href="https://github.com/usnistgov/OSCAL/issues/206">needed</a> to support the declaration of a rule language and value.</p>
-            </remarks>
-         </fields>
-         <assemblies ref="guideline">
-            <description>Additional recommendations for the use of the parameter, or around what values should be provided.</description>
-         </assemblies>
+         <field ref="usage" max-occurs="unbounded"/>
+         <field ref="constraint" max-occurs="unbounded"/>
+         <assembly ref="guideline" max-occurs="unbounded"/>
          <choice>
             <field ref="value">
                <description>A recommended parameter value or set of values.</description>
@@ -209,9 +195,7 @@
                </remarks>
             </assembly>
          </choice>
-         <fields ref="link">
-            <description/>
-         </fields>
+         <field ref="link" max-occurs="unbounded"/>
          <any/>
       </model>
       <remarks>
@@ -230,7 +214,7 @@
   <define-field name="usage" group-as="descriptions" as="mixed">
       <formal-name>Parameter description</formal-name>
       <description>Indicates and explains the purpose and use of a parameter</description>
-      <flag datatype="ID" ref="id"/>
+      <flag ref="id" datatype="ID"/>
    </define-field>
 
   <define-field name="constraint" group-as="constraints">
@@ -267,7 +251,7 @@
       <description>Presenting a choice among alternatives</description>
       <flag ref="how-many"/>
       <model>
-         <fields ref="choice"/>
+         <field ref="choice" max-occurs="unbounded"/>
          <any/>
       </model>
    </define-assembly>
@@ -280,16 +264,16 @@
   <define-assembly name="part" group-as="parts">
       <formal-name>Part</formal-name>
       <description>A partition or component of a control, subcontrol or part</description>
-      <flag datatype="ID" ref="id"/>
-      <flag required="yes" ref="name"/>
+      <flag ref="id" datatype="ID"/>
+      <flag ref="name" required="yes"/>
       <flag ref="ns"/>
       <flag ref="class"/>
       <model>
          <field ref="title"/>
-         <fields ref="prop"/>
+         <field ref="prop" max-occurs="unbounded"/>
          <prose/>
-         <assemblies ref="part"/>
-         <fields ref="link"/>
+         <assembly ref="part" max-occurs="unbounded"/>
+         <field ref="link" max-occurs="unbounded"/>
          <any/>
       </model>
       <remarks>
@@ -308,17 +292,17 @@
       </example>
    </define-assembly>
 
-   <define-flag name="test" datatype="string">
+  <define-flag name="test" datatype="string">
       <formal-name>Constraint test</formal-name>
       <description>A formal (executable) expression of a constraint</description>
    </define-flag>
-   
-   <define-flag name="how-many" datatype="string">
+
+  <define-flag name="how-many" datatype="string">
       <formal-name>Cardinality</formal-name>
       <description>When selecting, a requirement such as one or more</description>
    </define-flag>
-   
-   <define-flag name="depends-on" datatype="IDREF">
+
+  <define-flag name="depends-on" datatype="IDREF">
       <formal-name>Depends on</formal-name>
       <description>Another parameter invoking this one</description>
    </define-flag>