Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for date and date-time without timezones #480

Closed
4 tasks
wendellpiez opened this issue Aug 29, 2019 · 3 comments
Closed
4 tasks

Add support for date and date-time without timezones #480

wendellpiez opened this issue Aug 29, 2019 · 3 comments

Comments

@wendellpiez
Copy link
Contributor

User Story:

As a Metaschema user I may have a need for datatypes not yet supported, including date-time (dateTime) and date, without time zone. We need to add them.

Goals:

New data types for time and date-time without time zone can be referenced and validated. Unit tests for datatypes demonstrate this support. Sample schemas and data show how they can be used.

Dependencies:

None known

Acceptance Criteria

  • New datatypes supporting date and date-time (without requiring time zone) can be demonstrated to work in testing
  • All applicable documentation has been updated including web site docs on datatypes
  • A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
  • The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.
@wendellpiez wendellpiez added this to the OSCAL 1.0 Milestone 2 milestone Aug 29, 2019
@wendellpiez wendellpiez self-assigned this Aug 29, 2019
wendellpiez added a commit to wendellpiez/OSCAL that referenced this issue Sep 5, 2019
@wendellpiez
Copy link
Contributor Author

wendellpiez commented Sep 5, 2019

Implemented in PR #484, with unit tests.

Incidentally in testing this I discovered JSON Schema regex patterns are not treated as anchored for evaluation. So all our regexes have start-string ^ and end-string $ anchors now.

to do: update documentation.

@wendellpiez
Copy link
Contributor Author

FWIW - a time zone is permitted but not required on the new 'date' and 'dateTime' types.

david-waltermire pushed a commit to david-waltermire/OSCAL that referenced this issue Sep 26, 2019
david-waltermire added a commit to david-waltermire/OSCAL that referenced this issue Sep 26, 2019
…ime zones (usnistgov#480), including unit tests"

This reverts commit c42686e.
david-waltermire added a commit to david-waltermire/OSCAL that referenced this issue Sep 26, 2019
…ime zones (usnistgov#480), including unit tests"

This reverts commit c42686e.
david-waltermire pushed a commit to david-waltermire/OSCAL that referenced this issue Oct 1, 2019
david-waltermire added a commit that referenced this issue Oct 1, 2019
* Removed unnecessary metaschema check on json-value-key

* Now implementing in-xml="with-wrapper | unwrapped" on markup multiline fields, with default `with-wrapper` and corresponding changes to catalog and profile metaschemas.

* Updating catalog and profile schemas for (prose) unwrapping behavior

* Renamed 'json-behavior' in Metaschema to 'in-json' for consistency with 'in-xml'.

* Updating syntax in metaschemas 'in-json' for 'json-behavior'

* Updating json converter generator to provide in-xml behavior 'GROUPED'

* Refining json converter production from Metaschema

* Repairing egregious syntax error

* Rectify bug introduced with new feature

* Adding support for 'date' and 'dateTime' datatypes, with no time zones (#480), including unit tests

* Improvements and corrections to datatypes docs

* More small repairs on datatypes descriptions (table markup)

* Built out validations of allowed (enumerated) values, including Metaschema validation checks on whether given allowed values conform to their nominal type.

* Straightening kinks in datatype metaschema meta-validation

* More special handling of recalcitrant datatypes, this time 'NMTOKENS' and 'IDREFS'

* Addressing #463, adding `id`, `class` and `title` to `group` in profiles.

* Schematron improvements; documentation revisions (removing references to subcontrols)

* New Schematron for SP800-53-style catalogs

* Removed using in implementation to new metaschema for storage. Started on example, which is still invalid

* More work on an example, which is still not valid

* refactored schematron processing

* more work on SSP example

* Refactored generate-schema.sh to allow a metaschema to be provided on the CLI

* refactored SSP model to be more flat, while allowing component references

* slight cleanup

* old example

* Fixed content generation

* updated metaschema to new specs

* fixed content errors

* added metaschema examples

* Updated metaschema example.

* Adding 'prop' and 'part' to Profile Metaschema per #463

* Added 'param' to profile 'group' model

* JSON converter now properly handling flags assigned to assemblies identified by json key

* Added 'sort-id' property to controls in SP800-53 catalog

* Added support for direct generation of schema and converters

* Minor updates to SSP example and metaschema

* Adding computer build metaschema examples

* Extending allowed-values support to fields; unit tests: #437

* More work on allowed-values, now in XSD with unit tests for #437

* Repairing small datatyping bug thanks to regression testing

* Further adjustment for allowed-values in docs

* Renamed unit tests, now working

* Adding allowed-values unit tests

* Addressing #452: cleaned up character handling in XML-to-JSON conversion. Fixed the bug this exposed in markdown->HTML conversion. Added some light unit testing.

* More adjustments in view of comments to #452

* Repairing more bugs in character handling in Markdown and plain text, with unit testing: #452

* Improvements to Schematron for SP800-53 #400

* For #452, modified copy-and-convert script to suppress character-level post-processing in generated JSON; added unit tests for URIs to detect issues with solidus characters '/'

* Edited docs, including datatypes page per #452 (regarding character escapes).

* Cleanup metaschema valdiation errors

* changed use of the ID and IDREF datatype to NCName

* Removed ID and IDREF datatypes

* refactored components out of ssp metaschema

* Updated SP800-53 catalog and NIST profiles to current model

* Added name to information type.

* Continued refinement of the SSP model

* Updated metadata syntax

* changing CM-6 constraint to guidance

* continued work on SSP

* refactored responsible party to metadata

* fixed validation issues

* Fixed overriding enumerated values

* Fixed content validation errors based on latest metaschema changes

* Completed full pass through ssp model

* added prop to resource

* Fixed broken links in Markdown files. Small adjustments to SSP example.

* FDixed relapath error caused when the generated schema directory doesn't exist. Also attempting a fix for CircleCI colorized escape codes causing extra characters to appear.

* Fixed validation issue with component metaschema

* attempting to fix extra characters in CircleCI colorized messages

* Updated enumerated values to remove values that should appear in a given context

* quick reordering of elements

* hardcode ascii color codes

* Fixing broken links in readmes; file name update for consistency

* added unit tests for bug

* fixed color escaping in scripts

* Removed bogus unit test and fixed a small cardinality issue in the SSP model

* Improvements to docs generation

* Added JSON value keys to fields with flags (that did not have them)

* Bug reduction in XSD production mainly related to datatypes and allowed values

* Fixing glitch in JSON converter generator

* Adjustments repairing Metaschema issues

* CSS tweak in docs (model maps to use USWDS font)

* Adding to Milestone 2 release notes (in progress)

* Change to parameter setting in profile model (#494, #288); release notes towards M2

* Small improvements to copy in release notes (draft)

* Updating metaschemas to latest syntax, with improved Metaschema Schematron and unit test tweakage

* Adjustments to flag unit test and release notes

* SSP Metaschema review and update

* cleanup of unused role-id field

* reflactored role-id

* updated documentation for release.

* cleanup of IDs

* Minor schema and metaschema improvements

* M1 to M2 content converter

* Merging local and upstream changes to json example

* Relaxing rule over 'ID' flag to accept any string not only XML NCName

* reorganizing content for release

* more file moves

* Fixed changed path

* Updated FedRAMP profiles and catalog to current models

* Bug fix to XML-to-JSON converter generator

* Fixed copy script

* Updated FedRAMP baselines and catalog to match new metaschema syntax

* fixed merge error

* fixed another merge error

* fixed spelling issues

* Fix from Wendell
@david-waltermire
Copy link
Contributor

This was completed in PR #492.

bradh pushed a commit to bradh/OSCAL that referenced this issue Dec 4, 2019
* Removed unnecessary metaschema check on json-value-key

* Now implementing in-xml="with-wrapper | unwrapped" on markup multiline fields, with default `with-wrapper` and corresponding changes to catalog and profile metaschemas.

* Updating catalog and profile schemas for (prose) unwrapping behavior

* Renamed 'json-behavior' in Metaschema to 'in-json' for consistency with 'in-xml'.

* Updating syntax in metaschemas 'in-json' for 'json-behavior'

* Updating json converter generator to provide in-xml behavior 'GROUPED'

* Refining json converter production from Metaschema

* Repairing egregious syntax error

* Rectify bug introduced with new feature

* Adding support for 'date' and 'dateTime' datatypes, with no time zones (usnistgov#480), including unit tests

* Improvements and corrections to datatypes docs

* More small repairs on datatypes descriptions (table markup)

* Built out validations of allowed (enumerated) values, including Metaschema validation checks on whether given allowed values conform to their nominal type.

* Straightening kinks in datatype metaschema meta-validation

* More special handling of recalcitrant datatypes, this time 'NMTOKENS' and 'IDREFS'

* Addressing usnistgov#463, adding `id`, `class` and `title` to `group` in profiles.

* Schematron improvements; documentation revisions (removing references to subcontrols)

* New Schematron for SP800-53-style catalogs

* Removed using in implementation to new metaschema for storage. Started on example, which is still invalid

* More work on an example, which is still not valid

* refactored schematron processing

* more work on SSP example

* Refactored generate-schema.sh to allow a metaschema to be provided on the CLI

* refactored SSP model to be more flat, while allowing component references

* slight cleanup

* old example

* Fixed content generation

* updated metaschema to new specs

* fixed content errors

* added metaschema examples

* Updated metaschema example.

* Adding 'prop' and 'part' to Profile Metaschema per usnistgov#463

* Added 'param' to profile 'group' model

* JSON converter now properly handling flags assigned to assemblies identified by json key

* Added 'sort-id' property to controls in SP800-53 catalog

* Added support for direct generation of schema and converters

* Minor updates to SSP example and metaschema

* Adding computer build metaschema examples

* Extending allowed-values support to fields; unit tests: usnistgov#437

* More work on allowed-values, now in XSD with unit tests for usnistgov#437

* Repairing small datatyping bug thanks to regression testing

* Further adjustment for allowed-values in docs

* Renamed unit tests, now working

* Adding allowed-values unit tests

* Addressing usnistgov#452: cleaned up character handling in XML-to-JSON conversion. Fixed the bug this exposed in markdown->HTML conversion. Added some light unit testing.

* More adjustments in view of comments to usnistgov#452

* Repairing more bugs in character handling in Markdown and plain text, with unit testing: usnistgov#452

* Improvements to Schematron for SP800-53 usnistgov#400

* For usnistgov#452, modified copy-and-convert script to suppress character-level post-processing in generated JSON; added unit tests for URIs to detect issues with solidus characters '/'

* Edited docs, including datatypes page per usnistgov#452 (regarding character escapes).

* Cleanup metaschema valdiation errors

* changed use of the ID and IDREF datatype to NCName

* Removed ID and IDREF datatypes

* refactored components out of ssp metaschema

* Updated SP800-53 catalog and NIST profiles to current model

* Added name to information type.

* Continued refinement of the SSP model

* Updated metadata syntax

* changing CM-6 constraint to guidance

* continued work on SSP

* refactored responsible party to metadata

* fixed validation issues

* Fixed overriding enumerated values

* Fixed content validation errors based on latest metaschema changes

* Completed full pass through ssp model

* added prop to resource

* Fixed broken links in Markdown files. Small adjustments to SSP example.

* FDixed relapath error caused when the generated schema directory doesn't exist. Also attempting a fix for CircleCI colorized escape codes causing extra characters to appear.

* Fixed validation issue with component metaschema

* attempting to fix extra characters in CircleCI colorized messages

* Updated enumerated values to remove values that should appear in a given context

* quick reordering of elements

* hardcode ascii color codes

* Fixing broken links in readmes; file name update for consistency

* added unit tests for bug

* fixed color escaping in scripts

* Removed bogus unit test and fixed a small cardinality issue in the SSP model

* Improvements to docs generation

* Added JSON value keys to fields with flags (that did not have them)

* Bug reduction in XSD production mainly related to datatypes and allowed values

* Fixing glitch in JSON converter generator

* Adjustments repairing Metaschema issues

* CSS tweak in docs (model maps to use USWDS font)

* Adding to Milestone 2 release notes (in progress)

* Change to parameter setting in profile model (usnistgov#494, usnistgov#288); release notes towards M2

* Small improvements to copy in release notes (draft)

* Updating metaschemas to latest syntax, with improved Metaschema Schematron and unit test tweakage

* Adjustments to flag unit test and release notes

* SSP Metaschema review and update

* cleanup of unused role-id field

* reflactored role-id

* updated documentation for release.

* cleanup of IDs

* Minor schema and metaschema improvements

* M1 to M2 content converter

* Merging local and upstream changes to json example

* Relaxing rule over 'ID' flag to accept any string not only XML NCName

* reorganizing content for release

* more file moves

* Fixed changed path

* Updated FedRAMP profiles and catalog to current models

* Bug fix to XML-to-JSON converter generator

* Fixed copy script

* Updated FedRAMP baselines and catalog to match new metaschema syntax

* fixed merge error

* fixed another merge error

* fixed spelling issues

* Fix from Wendell
aj-stein-nist pushed a commit to aj-stein-nist/OSCAL-forked that referenced this issue Jan 25, 2023
* Removed unnecessary metaschema check on json-value-key

* Now implementing in-xml="with-wrapper | unwrapped" on markup multiline fields, with default `with-wrapper` and corresponding changes to catalog and profile metaschemas.

* Updating catalog and profile schemas for (prose) unwrapping behavior

* Renamed 'json-behavior' in Metaschema to 'in-json' for consistency with 'in-xml'.

* Updating syntax in metaschemas 'in-json' for 'json-behavior'

* Updating json converter generator to provide in-xml behavior 'GROUPED'

* Refining json converter production from Metaschema

* Repairing egregious syntax error

* Rectify bug introduced with new feature

* Adding support for 'date' and 'dateTime' datatypes, with no time zones (usnistgov#480), including unit tests

* Improvements and corrections to datatypes docs

* More small repairs on datatypes descriptions (table markup)

* Built out validations of allowed (enumerated) values, including Metaschema validation checks on whether given allowed values conform to their nominal type.

* Straightening kinks in datatype metaschema meta-validation

* More special handling of recalcitrant datatypes, this time 'NMTOKENS' and 'IDREFS'

* Addressing usnistgov#463, adding `id`, `class` and `title` to `group` in profiles.

* Schematron improvements; documentation revisions (removing references to subcontrols)

* New Schematron for SP800-53-style catalogs

* Removed using in implementation to new metaschema for storage. Started on example, which is still invalid

* More work on an example, which is still not valid

* refactored schematron processing

* more work on SSP example

* Refactored generate-schema.sh to allow a metaschema to be provided on the CLI

* refactored SSP model to be more flat, while allowing component references

* slight cleanup

* old example

* Fixed content generation

* updated metaschema to new specs

* fixed content errors

* added metaschema examples

* Updated metaschema example.

* Adding 'prop' and 'part' to Profile Metaschema per usnistgov#463

* Added 'param' to profile 'group' model

* JSON converter now properly handling flags assigned to assemblies identified by json key

* Added 'sort-id' property to controls in SP800-53 catalog

* Added support for direct generation of schema and converters

* Minor updates to SSP example and metaschema

* Adding computer build metaschema examples

* Extending allowed-values support to fields; unit tests: usnistgov#437

* More work on allowed-values, now in XSD with unit tests for usnistgov#437

* Repairing small datatyping bug thanks to regression testing

* Further adjustment for allowed-values in docs

* Renamed unit tests, now working

* Adding allowed-values unit tests

* Addressing usnistgov#452: cleaned up character handling in XML-to-JSON conversion. Fixed the bug this exposed in markdown->HTML conversion. Added some light unit testing.

* More adjustments in view of comments to usnistgov#452

* Repairing more bugs in character handling in Markdown and plain text, with unit testing: usnistgov#452

* Improvements to Schematron for SP800-53 #400

* For usnistgov#452, modified copy-and-convert script to suppress character-level post-processing in generated JSON; added unit tests for URIs to detect issues with solidus characters '/'

* Edited docs, including datatypes page per usnistgov#452 (regarding character escapes).

* Cleanup metaschema valdiation errors

* changed use of the ID and IDREF datatype to NCName

* Removed ID and IDREF datatypes

* refactored components out of ssp metaschema

* Updated SP800-53 catalog and NIST profiles to current model

* Added name to information type.

* Continued refinement of the SSP model

* Updated metadata syntax

* changing CM-6 constraint to guidance

* continued work on SSP

* refactored responsible party to metadata

* fixed validation issues

* Fixed overriding enumerated values

* Fixed content validation errors based on latest metaschema changes

* Completed full pass through ssp model

* added prop to resource

* Fixed broken links in Markdown files. Small adjustments to SSP example.

* FDixed relapath error caused when the generated schema directory doesn't exist. Also attempting a fix for CircleCI colorized escape codes causing extra characters to appear.

* Fixed validation issue with component metaschema

* attempting to fix extra characters in CircleCI colorized messages

* Updated enumerated values to remove values that should appear in a given context

* quick reordering of elements

* hardcode ascii color codes

* Fixing broken links in readmes; file name update for consistency

* added unit tests for bug

* fixed color escaping in scripts

* Removed bogus unit test and fixed a small cardinality issue in the SSP model

* Improvements to docs generation

* Added JSON value keys to fields with flags (that did not have them)

* Bug reduction in XSD production mainly related to datatypes and allowed values

* Fixing glitch in JSON converter generator

* Adjustments repairing Metaschema issues

* CSS tweak in docs (model maps to use USWDS font)

* Adding to Milestone 2 release notes (in progress)

* Change to parameter setting in profile model (usnistgov#494, #288); release notes towards M2

* Small improvements to copy in release notes (draft)

* Updating metaschemas to latest syntax, with improved Metaschema Schematron and unit test tweakage

* Adjustments to flag unit test and release notes

* SSP Metaschema review and update

* cleanup of unused role-id field

* reflactored role-id

* updated documentation for release.

* cleanup of IDs

* Minor schema and metaschema improvements

* M1 to M2 content converter

* Merging local and upstream changes to json example

* Relaxing rule over 'ID' flag to accept any string not only XML NCName

* reorganizing content for release

* more file moves

* Fixed changed path

* Updated FedRAMP profiles and catalog to current models

* Bug fix to XML-to-JSON converter generator

* Fixed copy script

* Updated FedRAMP baselines and catalog to match new metaschema syntax

* fixed merge error

* fixed another merge error

* fixed spelling issues

* Fix from Wendell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants