From e7a0f8dc2192c409c277903ffe58438e1b40ca10 Mon Sep 17 00:00:00 2001 From: Stephen Brawner Date: Sat, 29 Feb 2020 20:08:44 -0800 Subject: [PATCH 1/6] First draft quality declaration Signed-off-by: Stephen Brawner --- QUALITY_DECLARATION.md | 112 +++++++++++++++++++++++++++++++++++++++++ README.md | 4 ++ 2 files changed, 116 insertions(+) create mode 100644 QUALITY_DECLARATION.md diff --git a/QUALITY_DECLARATION.md b/QUALITY_DECLARATION.md new file mode 100644 index 0000000..47950f7 --- /dev/null +++ b/QUALITY_DECLARATION.md @@ -0,0 +1,112 @@ +This document is a declaration of software quality for the `rcpputils` package, based on the guidelines in [REP-2004](https://www.ros.org/reps/rep-2004.html). + +# `rcpputils` Quality Declaration + +The package `rcpputils` claims to be in the **Quality Level 1** category. + +Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 1 in REP-2004](https://www.ros.org/reps/rep-2004.html). + +## Version Policy + +### Version Scheme + +`rcpputils` uses `semver` according to the recommendation for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#versioning), and is at or above a stable version, i.e. `>= 1.0.0`. + +### API Stability Within a Released ROS Distribution + +`rcpputils` will not break public API within a released ROS distribution, i.e. no major releases once the ROS distribution is released. + +### ABI Stability Within a Released ROS Distribution + +`rcpputils` contains C code and therefore must be concerned with ABI stability, and will maintain ABI stability within a ROS distribution. + +### Public API Declaration + +All symbols in the installed headers are considered part of the public API. + +All installed headers are in the `include` directory of the package, headers in any other folders are not installed and considered private. + +## Change Control Process + +`rcpputils` follows the recommended guidelines for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-requirements). + +This includes: + +- all changes occur through a pull request +- all pull request have two peer reviews +- all pull request must pass CI on all [tier 1 platforms](https://www.ros.org/reps/rep-2000.html#support-tiers) +- all pull request must resolve related documentation changes before merging + +## Documentation + +### Feature Documentation + +`rcpputils` has a [feature list](docs/FEATURES.md) and each item in the list links to the corresponding feature documentation. +There is documentation for all of the features, and new features require documentation before being added. + +### Public API Documentation + +TODO upload docs + +`rcpputils` has embedded API documentation and it is generated using doxygen and is [hosted](https://docs.ros2.org/eloquent/api/rcpputils/index.html) along side the feature documentation. +There is documentation for all of the public API, and new additions to the public API require documentation before being added. + +### License + +The license for `rcpputils` is Apache 2.0, and a summary is in each source file, the type is declared in the `package.xml` manifest file, and a full copy of the license is in the `LICENSE` file. + +There is an automated test which runs a linter that ensures each file has a license statement. + +### Copyright Statements + +The copyright holders each provide a statement of copyright in each source code file in `rcpputils`. + +There is an automated test which runs a linter that ensures each file has at least one copyright statement. + +## Testing + +### Feature Testing + +Each feature in `rcpputils` has corresponding tests which simulate typical usage, and they are located in the `test` directory. +New features are required to have tests before being added. + +### Public API Testing + +Each part of the public API have tests, and new additions or changes to the public API require tests before being added. +The tests aim to cover both typical usage and corner cases, but are quantified by contributing to code coverage. + +### Coverage + +`rcpputils` follows the recommendations for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#test-coverage), and opts to use branch coverage instead of line coverage. + +This includes: + +- tracking and reporting branch coverage statistics +- achieving and maintaining branch coverage at or above 95% +- no lines are manually skipped in coverage calculations + +Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers. + +Current coverage statistics can be viewed on [codecov.io](https://codecov.io/gh/j-rivero/rcpputils): + +### Performance + +TODO document performance tests + +`rcpputils` follows the recommendations for performance testing of C++ code in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-requirements), and opts to do performance analysis on each release rather than each change. + +### Linters and Static Analysis + +`rcpputils` uses and passes all the standard linters and static analysis tools for a C++ package as described in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#linters). + +## Dependencies + +`rcpputils` has the following ROS dependencies: +* `rcutils` + +It has several "buildtool" dependencies, which do not affect the resulting quality of the package, because they do not contribute to the public library API. +It also has several test dependencies, which do not affect the resulting quality of the package, because they are only used to build and run the test code. + +## Platform Support + +`rcpputils` supports all of the tier 1 platforms as described in [REP-2000](https://www.ros.org/reps/rep-2000.html#support-tiers), and tests each change against all of them. diff --git a/README.md b/README.md index df4f7cf..8340ed3 100644 --- a/README.md +++ b/README.md @@ -5,6 +5,10 @@ `rcpputils` is a C++ API consisting of macros, functions, and data structures intended for use throughout the ROS 2 codebase +## Quality Declaration + +This package claims to be in the **Quality Level 1** category, see the [Quality Declaration](QUALITY_DECLARATION.md) for more details. + ## API This package currently contains: * Assertion functions From 2851821b8960f5e1ec2e9c65e1726ee0a8f09af6 Mon Sep 17 00:00:00 2001 From: Stephen Brawner Date: Tue, 21 Apr 2020 17:32:02 -0700 Subject: [PATCH 2/6] QD now describes current status Signed-off-by: Stephen Brawner --- QUALITY_DECLARATION.md | 116 +++++++++++++++++++++++------------------ 1 file changed, 66 insertions(+), 50 deletions(-) diff --git a/QUALITY_DECLARATION.md b/QUALITY_DECLARATION.md index 47950f7..b8b0c0f 100644 --- a/QUALITY_DECLARATION.md +++ b/QUALITY_DECLARATION.md @@ -2,111 +2,127 @@ This document is a declaration of software quality for the `rcpputils` package, # `rcpputils` Quality Declaration -The package `rcpputils` claims to be in the **Quality Level 1** category. +The package `rcpputils` claims to be in the **Quality Level 4** category. -Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 1 in REP-2004](https://www.ros.org/reps/rep-2004.html). +Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 4 in REP-2004](https://www.ros.org/reps/rep-2004.html). -## Version Policy +## Version Policy [1] -### Version Scheme +### Version Scheme [1.i] -`rcpputils` uses `semver` according to the recommendation for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#versioning), and is at or above a stable version, i.e. `>= 1.0.0`. +`rcpputils` uses `semver` according to the recommendation for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#versioning). -### API Stability Within a Released ROS Distribution +### Version Stability [1.ii] -`rcpputils` will not break public API within a released ROS distribution, i.e. no major releases once the ROS distribution is released. - -### ABI Stability Within a Released ROS Distribution +`rcpputils` is not yet at a stable version, i.e. `>= 1.0.0`. -`rcpputils` contains C code and therefore must be concerned with ABI stability, and will maintain ABI stability within a ROS distribution. -### Public API Declaration +### Public API Declaration [1.iii] All symbols in the installed headers are considered part of the public API. All installed headers are in the `include` directory of the package, headers in any other folders are not installed and considered private. -## Change Control Process +### API Stability Within a Released ROS Distribution [1.iv]/[1.vi] + +`rcpputils` will not break public API within a released ROS distribution, i.e. no major releases once the ROS distribution is released. + +### ABI Stability Within a Released ROS Distribution [1.v]/[1.vi] + +`rcpputils` contains C code and therefore must be concerned with ABI stability, and will maintain ABI stability within a ROS distribution. + +## Change Control Process [2] `rcpputils` follows the recommended guidelines for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-requirements). -This includes: +### Change Requests [2.i] + +This package requires that all changes occur through a pull request, and all pull requests must resolve related documentation changes before merging. + +### Contributor Origin [2.ii] + This package has a confirmation of contributor origin policy, which can be found in [CONTRIBUTING](../CONTRIBUTING.md) + +### Peer Review Policy [2.iii] + + Following the recommended guidelines for ROS Core packages, all pull requests must have at least 1 peer review. + +### Continuous Integration [2.iv] -- all changes occur through a pull request -- all pull request have two peer reviews -- all pull request must pass CI on all [tier 1 platforms](https://www.ros.org/reps/rep-2000.html#support-tiers) -- all pull request must resolve related documentation changes before merging +All pull request must pass CI on all [tier 1 platforms](https://www.ros.org/reps/rep-2000.html#support-tiers) -## Documentation +## Documentation [3] -### Feature Documentation +### Feature Documentation [3.i] `rcpputils` has a [feature list](docs/FEATURES.md) and each item in the list links to the corresponding feature documentation. There is documentation for all of the features, and new features require documentation before being added. -### Public API Documentation +### Public API Documentation [3.ii] -TODO upload docs +`rcpputils` has embedded API documentation. It is not yet hosted publicly. -`rcpputils` has embedded API documentation and it is generated using doxygen and is [hosted](https://docs.ros2.org/eloquent/api/rcpputils/index.html) along side the feature documentation. -There is documentation for all of the public API, and new additions to the public API require documentation before being added. +### License [3.iii] -### License - -The license for `rcpputils` is Apache 2.0, and a summary is in each source file, the type is declared in the `package.xml` manifest file, and a full copy of the license is in the `LICENSE` file. +The license for `rcpputils` is Apache 2.0, and a summary is in each source file, the type is declared in the `package.xml` manifest file, and a full copy of the license is in the [LICENSE](./LICENSE) file. There is an automated test which runs a linter that ensures each file has a license statement. -### Copyright Statements +Most recent test results can be found [here](http://build.ros2.org/view/Epr/job/Epr__rcpputils__ubuntu_bionic_amd64/lastBuild/testReport/rcpputils/) + +### Copyright Statements [3.iv] The copyright holders each provide a statement of copyright in each source code file in `rcpputils`. There is an automated test which runs a linter that ensures each file has at least one copyright statement. -## Testing +## Testing [4] -### Feature Testing +### Feature Testing [4.i] Each feature in `rcpputils` has corresponding tests which simulate typical usage, and they are located in the `test` directory. New features are required to have tests before being added. -### Public API Testing +### Public API Testing [4.ii] Each part of the public API have tests, and new additions or changes to the public API require tests before being added. The tests aim to cover both typical usage and corner cases, but are quantified by contributing to code coverage. -### Coverage - -`rcpputils` follows the recommendations for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#test-coverage), and opts to use branch coverage instead of line coverage. - -This includes: - -- tracking and reporting branch coverage statistics -- achieving and maintaining branch coverage at or above 95% -- no lines are manually skipped in coverage calculations - -Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers. - -Current coverage statistics can be viewed on [codecov.io](https://codecov.io/gh/j-rivero/rcpputils): +### Coverage [4.iii] -### Performance +`rcpputils` does not currently track test coverage. -TODO document performance tests +### Performance [4.iv] -`rcpputils` follows the recommendations for performance testing of C++ code in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-requirements), and opts to do performance analysis on each release rather than each change. +`rcpputils` does not currently have performance tests. -### Linters and Static Analysis +### Linters and Static Analysis [4.v] `rcpputils` uses and passes all the standard linters and static analysis tools for a C++ package as described in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#linters). -## Dependencies +## Dependencies [5] -`rcpputils` has the following ROS dependencies: +### Direct Runtime ROS Dependencies [5.i]/[5.ii] + +`rcpputils` has the following runtime ROS dependencies: * `rcutils` It has several "buildtool" dependencies, which do not affect the resulting quality of the package, because they do not contribute to the public library API. It also has several test dependencies, which do not affect the resulting quality of the package, because they are only used to build and run the test code. -## Platform Support +### Direct Runtime Non-ROS Dependencies [5.iii] + +`rcpputils` does not have any runtime non-ROS dependencies. + +## Platform Support [6] `rcpputils` supports all of the tier 1 platforms as described in [REP-2000](https://www.ros.org/reps/rep-2000.html#support-tiers), and tests each change against all of them. + +Currently nightly results can be seen here: +* [linux-aarch64_release](https://ci.ros2.org/view/nightly/job/nightly_linux-aarch64_release/lastBuild/testReport/rcpputils/) +* [linux-arm64_release](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rcpputils/) +* [mac_osx_release](https://ci.ros2.org/view/nightly/job/nightly_osx_release/lastBuild/testReport/rcpputils/) +* [windows_release](https://ci.ros2.org/view/nightly/job/nightly_win_rel/lastBuild/testReport/rcpputils/) + +## Vulnerability Disclosure Policy [7.i] + +This package does not yet have a Vulnerability Disclosure Policy From bfc95f3b4e0df32ec5eacd42d29fcace43d35c8e Mon Sep 17 00:00:00 2001 From: Stephen Brawner Date: Thu, 23 Apr 2020 15:39:32 -0700 Subject: [PATCH 3/6] Fixup Signed-off-by: Stephen Brawner --- QUALITY_DECLARATION.md | 10 +++++++--- README.md | 2 +- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/QUALITY_DECLARATION.md b/QUALITY_DECLARATION.md index b8b0c0f..126c698 100644 --- a/QUALITY_DECLARATION.md +++ b/QUALITY_DECLARATION.md @@ -16,7 +16,6 @@ Below are the rationales, notes, and caveats for this claim, organized by each r `rcpputils` is not yet at a stable version, i.e. `>= 1.0.0`. - ### Public API Declaration [1.iii] All symbols in the installed headers are considered part of the public API. @@ -37,10 +36,11 @@ All installed headers are in the `include` directory of the package, headers in ### Change Requests [2.i] -This package requires that all changes occur through a pull request, and all pull requests must resolve related documentation changes before merging. +This package requires that all changes occur through a pull request. ### Contributor Origin [2.ii] - This package has a confirmation of contributor origin policy, which can be found in [CONTRIBUTING](../CONTRIBUTING.md) + + This package uses DCO as its confirmation of contributor origin policy. More information can be found in [CONTRIBUTING](../CONTRIBUTING.md). ### Peer Review Policy [2.iii] @@ -50,6 +50,10 @@ This package requires that all changes occur through a pull request, and all pul All pull request must pass CI on all [tier 1 platforms](https://www.ros.org/reps/rep-2000.html#support-tiers) +### Documentation Policy [2.v] + +All pull requests must resolve related documentation changes before merging. + ## Documentation [3] ### Feature Documentation [3.i] diff --git a/README.md b/README.md index 8340ed3..4ba783d 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ ## Quality Declaration -This package claims to be in the **Quality Level 1** category, see the [Quality Declaration](QUALITY_DECLARATION.md) for more details. +This package claims to be in the **Quality Level 4** category, see the [Quality Declaration](QUALITY_DECLARATION.md) for more details. ## API This package currently contains: From f667481824f29260f2e056238cee925105ac4ed0 Mon Sep 17 00:00:00 2001 From: Stephen Brawner Date: Thu, 23 Apr 2020 16:34:55 -0700 Subject: [PATCH 4/6] Fix link Signed-off-by: Stephen Brawner --- QUALITY_DECLARATION.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/QUALITY_DECLARATION.md b/QUALITY_DECLARATION.md index 126c698..47f30b7 100644 --- a/QUALITY_DECLARATION.md +++ b/QUALITY_DECLARATION.md @@ -123,7 +123,7 @@ It also has several test dependencies, which do not affect the resulting quality Currently nightly results can be seen here: * [linux-aarch64_release](https://ci.ros2.org/view/nightly/job/nightly_linux-aarch64_release/lastBuild/testReport/rcpputils/) -* [linux-arm64_release](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rcpputils/) +* [linux_release](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rcpputils/) * [mac_osx_release](https://ci.ros2.org/view/nightly/job/nightly_osx_release/lastBuild/testReport/rcpputils/) * [windows_release](https://ci.ros2.org/view/nightly/job/nightly_win_rel/lastBuild/testReport/rcpputils/) From eb13e87af4cae8e3f5c9127ecdad60a052165cb1 Mon Sep 17 00:00:00 2001 From: Stephen Brawner Date: Fri, 24 Apr 2020 12:52:10 -0700 Subject: [PATCH 5/6] Fixup Signed-off-by: Stephen Brawner --- QUALITY_DECLARATION.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/QUALITY_DECLARATION.md b/QUALITY_DECLARATION.md index 47f30b7..5dff29b 100644 --- a/QUALITY_DECLARATION.md +++ b/QUALITY_DECLARATION.md @@ -40,7 +40,7 @@ This package requires that all changes occur through a pull request. ### Contributor Origin [2.ii] - This package uses DCO as its confirmation of contributor origin policy. More information can be found in [CONTRIBUTING](../CONTRIBUTING.md). + This package uses DCO as its confirmation of contributor origin policy. More information can be found in [CONTRIBUTING](CONTRIBUTING.md). ### Peer Review Policy [2.iii] @@ -79,6 +79,8 @@ The copyright holders each provide a statement of copyright in each source code There is an automated test which runs a linter that ensures each file has at least one copyright statement. +The results of the test can be found [here](http://build.ros2.org/view/Epr/job/Epr__rcpputils__ubuntu_bionic_amd64/lastBuild/testReport/rcpputils/copyright/). + ## Testing [4] ### Feature Testing [4.i] @@ -103,6 +105,8 @@ The tests aim to cover both typical usage and corner cases, but are quantified b `rcpputils` uses and passes all the standard linters and static analysis tools for a C++ package as described in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#linters). +Results of the nightly linter tests can be found [here](http://build.ros2.org/view/Epr/job/Epr__rcpputils__ubuntu_bionic_amd64/lastBuild/testReport/rcpputils/). + ## Dependencies [5] ### Direct Runtime ROS Dependencies [5.i]/[5.ii] From 42645d99ace1bf012fed38d861b91b2a938107f4 Mon Sep 17 00:00:00 2001 From: brawner Date: Mon, 27 Apr 2020 09:58:08 -0700 Subject: [PATCH 6/6] Update QUALITY_DECLARATION.md Signed-off-by: Stephen Brawner --- QUALITY_DECLARATION.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/QUALITY_DECLARATION.md b/QUALITY_DECLARATION.md index 5dff29b..c369178 100644 --- a/QUALITY_DECLARATION.md +++ b/QUALITY_DECLARATION.md @@ -44,7 +44,7 @@ This package requires that all changes occur through a pull request. ### Peer Review Policy [2.iii] - Following the recommended guidelines for ROS Core packages, all pull requests must have at least 1 peer review. +Following the recommended guidelines for ROS Core packages, all pull requests must have at least 1 peer review. ### Continuous Integration [2.iv]