Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue343 roundtrip fixes #410

Merged

Conversation

david-waltermire
Copy link
Contributor

@david-waltermire david-waltermire commented Jun 6, 2019

Committer Notes

Fixed comparison from PR #405 addressing issue #343.

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you squashed any non-relevant commits and commit messages? [instructions]

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you included examples of how to use your new feature(s)?

@david-waltermire david-waltermire changed the title Issue343 rooundtrip fixes Issue343 roundtrip fixes Jun 6, 2019
@david-waltermire david-waltermire force-pushed the issue343-rooundtrip-fixes branch 3 times, most recently from 72a4839 to bf6f70b Compare June 6, 2019 04:51
@david-waltermire david-waltermire force-pushed the issue343-rooundtrip-fixes branch 2 times, most recently from c8a7f10 to 80974d1 Compare June 6, 2019 11:28
@wendellpiez wendellpiez mentioned this pull request Jun 7, 2019
6 tasks
howieavp76 and others added 11 commits June 11, 2019 05:47
Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Remove Unnecessary Python Libraries

No longer using several external libraries

Minor Updates

FIxed git ignores
Changed from -v check to something more compatible

Updates to Testing

Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Minor Updates

FIxed git ignores
Changed from -v check to something more compatible

Bash Generalization

Fixed the -v statements to use a more portable -z statement.

Added Exit Code Logic to XML Round Trip Comparison

Refactor XML Comparison

Removes XMLDiff

Adds exit code to the Python code, does error count tracking

Test Content Converer Fix

XML RoundTrip in Run All Tests

Round Trip XML->JSON->XML

Working for all NIST 800-53 checks (FedRAMP currently bombing)

Color Output

Printf messages are now color coded for improved readability in large log files

Temp Directory

Round trip files now written to the Temp directory, this directory added to build artifacts in CI/CD

Rename Temporal Files

Build files for roundtrip now carry their base naming conventions for troubleshooting

Add Roundtrip Check to CI/CD

Initial Attempt

Fix CI/CD Bug

Typo on base image

Local JAR

Trying to get it working end to end with local file

Dynamic Paths

Fix CI/CD bug finding Python/Saxon scripts

PIP Installs

Python Import Error Fix

UTF Encoding

Refactor Python

Get etree.parse to work in CI/CD

Shorten FileNames

Build Artifacts

Use the supplied build directory

Troubleshoot Artifacts

Dynamic JAR

Dynamic JAR 2

Maven Install

Adds Maven to the Circle CI image

Maven Attempt #2

Class Path Refactor

Refactor CI/CD

Faster build time

CI/CD Bug Fix

Fix Path Error

Copy Build Artifacts

Copy Artifacts Attempt #2

Artifact Try #3

Artifact Attempt #4

Copy Step

Artifacts #5

Fix Tab

Artifact #6

Artifact #7

Artifact #8

Artifact #9

Initial JSON Roundtrip

Remove Diff

Updates to Testing

Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Minor Updates

FIxed git ignores
Changed from -v check to something more compatible

Updates to Testing

Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Bash Generalization

Fixed the -v statements to use a more portable -z statement.

Added Exit Code Logic to XML Round Trip Comparison

Refactor XML Comparison

Removes XMLDiff

Adds exit code to the Python code, does error count tracking

Test Content Converer Fix

XML RoundTrip in Run All Tests

Round Trip XML->JSON->XML

Working for all NIST 800-53 checks (FedRAMP currently bombing)

Color Output

Printf messages are now color coded for improved readability in large log files

Temp Directory

Round trip files now written to the Temp directory, this directory added to build artifacts in CI/CD

Rename Temporal Files

Build files for roundtrip now carry their base naming conventions for troubleshooting

Add Roundtrip Check to CI/CD

Initial Attempt

Fix CI/CD Bug

Typo on base image

Local JAR

Trying to get it working end to end with local file

Dynamic Paths

Fix CI/CD bug finding Python/Saxon scripts

PIP Installs

Python Import Error Fix

UTF Encoding

Refactor Python

Get etree.parse to work in CI/CD

Shorten FileNames

Build Artifacts

Use the supplied build directory

Troubleshoot Artifacts

Dynamic JAR

Dynamic JAR 2

Maven Install

Adds Maven to the Circle CI image

Maven Attempt #2

Class Path Refactor

CI/CD Bug Fix

Fix Path Error

Copy Build Artifacts

Copy Artifacts Attempt #2

Artifact Try #3

Artifact Attempt #4

Copy Step

Artifact #7

Artifact #8

Artifact #9

Initial JSON Roundtrip

Remove Diff

Merge with Master

CI/CD Fix #2

CI/CD Fix #4

CI/CD Fix #5

CI/CD Fix #6
- Added file naming conventions for XSL conversion templates
- Renamed schema files to follow the naming guidance
- Reorganized metaschema docuemntation
- Refactored scripts to produce files using correct names
- Fixed broken links in repo and website
- Fixed spelling errors and broken links
- Added markdown-link-checker to CI process to check PRs for broken links
- Added colorization to many build messages
- Added support for JSON min and YAML content generation from JSON files (using prettyprint)
… names.

Added a validation step for the round-tripped file.
* Small bug fixes in scripts

* Getting closer to round tripping catalogs
@david-waltermire david-waltermire merged commit 25f3575 into usnistgov:master Jun 11, 2019
david-waltermire pushed a commit that referenced this pull request Jun 11, 2019
@david-waltermire david-waltermire deleted the issue343-rooundtrip-fixes branch June 15, 2019 15:00
aj-stein-nist referenced this pull request in aj-stein-nist/OSCAL-forked Jan 25, 2023
- Implemented Round Trip testing of XML->JSON->XML on content in CI/CD workflow.
- Repaired error in catalog and profile metaschemas, with Schematron checks to detect these errors.
- Removing whitespace stripping in converted content
- Repairing bug in round trip handling of quote marks " (#7)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants