From 0761e7da39960c4e7125b51b9e29c20bf8bbb706 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Wed, 16 Feb 2022 15:51:08 +0100 Subject: [PATCH 01/12] iox-#743 Move 'examples' section and rename it to 'tutorials' Signed-off-by: Simon Hoinkis --- doc/website/.pages | 3 ++- doc/website/{getting-started/examples => tutorials}/.pages | 0 .../{getting-started/examples => tutorials}/callbacks.md | 0 .../{getting-started/examples => tutorials}/callbacks_in_c.md | 0 .../{getting-started/examples => tutorials}/complexdata.md | 0 .../examples => tutorials}/ice_access_control.md | 0 .../{getting-started/examples => tutorials}/icecrystal.md | 0 .../{getting-started/examples => tutorials}/icedelivery.md | 0 .../examples => tutorials}/icedelivery_in_c.md | 0 .../{getting-started/examples => tutorials}/icedocker.md | 0 .../{getting-started/examples => tutorials}/iceensemble.md | 0 .../{getting-started/examples => tutorials}/icehello.md | 0 .../{getting-started/examples => tutorials}/iceoptions.md | 0 doc/website/{getting-started/examples => tutorials}/iceperf.md | 0 doc/website/{getting-started/examples => tutorials}/index.md | 0 .../{getting-started/examples => tutorials}/singleprocess.md | 0 .../{getting-started/examples => tutorials}/user_header.md | 0 doc/website/{getting-started/examples => tutorials}/waitset.md | 0 .../{getting-started/examples => tutorials}/waitset_in_c.md | 0 19 files changed, 2 insertions(+), 1 deletion(-) rename doc/website/{getting-started/examples => tutorials}/.pages (100%) rename doc/website/{getting-started/examples => tutorials}/callbacks.md (100%) rename doc/website/{getting-started/examples => tutorials}/callbacks_in_c.md (100%) rename doc/website/{getting-started/examples => tutorials}/complexdata.md (100%) rename doc/website/{getting-started/examples => tutorials}/ice_access_control.md (100%) rename doc/website/{getting-started/examples => tutorials}/icecrystal.md (100%) rename doc/website/{getting-started/examples => tutorials}/icedelivery.md (100%) rename doc/website/{getting-started/examples => tutorials}/icedelivery_in_c.md (100%) rename doc/website/{getting-started/examples => tutorials}/icedocker.md (100%) rename doc/website/{getting-started/examples => tutorials}/iceensemble.md (100%) rename doc/website/{getting-started/examples => tutorials}/icehello.md (100%) rename doc/website/{getting-started/examples => tutorials}/iceoptions.md (100%) rename doc/website/{getting-started/examples => tutorials}/iceperf.md (100%) rename doc/website/{getting-started/examples => tutorials}/index.md (100%) rename doc/website/{getting-started/examples => tutorials}/singleprocess.md (100%) rename doc/website/{getting-started/examples => tutorials}/user_header.md (100%) rename doc/website/{getting-started/examples => tutorials}/waitset.md (100%) rename doc/website/{getting-started/examples => tutorials}/waitset_in_c.md (100%) diff --git a/doc/website/.pages b/doc/website/.pages index c555be266a..3d52d8d0c1 100644 --- a/doc/website/.pages +++ b/doc/website/.pages @@ -1,6 +1,7 @@ nav: - index.md - getting-started - - API-reference + - tutorials - FAQ.md - advanced + - API-reference diff --git a/doc/website/getting-started/examples/.pages b/doc/website/tutorials/.pages similarity index 100% rename from doc/website/getting-started/examples/.pages rename to doc/website/tutorials/.pages diff --git a/doc/website/getting-started/examples/callbacks.md b/doc/website/tutorials/callbacks.md similarity index 100% rename from doc/website/getting-started/examples/callbacks.md rename to doc/website/tutorials/callbacks.md diff --git a/doc/website/getting-started/examples/callbacks_in_c.md b/doc/website/tutorials/callbacks_in_c.md similarity index 100% rename from doc/website/getting-started/examples/callbacks_in_c.md rename to doc/website/tutorials/callbacks_in_c.md diff --git a/doc/website/getting-started/examples/complexdata.md b/doc/website/tutorials/complexdata.md similarity index 100% rename from doc/website/getting-started/examples/complexdata.md rename to doc/website/tutorials/complexdata.md diff --git a/doc/website/getting-started/examples/ice_access_control.md b/doc/website/tutorials/ice_access_control.md similarity index 100% rename from doc/website/getting-started/examples/ice_access_control.md rename to doc/website/tutorials/ice_access_control.md diff --git a/doc/website/getting-started/examples/icecrystal.md b/doc/website/tutorials/icecrystal.md similarity index 100% rename from doc/website/getting-started/examples/icecrystal.md rename to doc/website/tutorials/icecrystal.md diff --git a/doc/website/getting-started/examples/icedelivery.md b/doc/website/tutorials/icedelivery.md similarity index 100% rename from doc/website/getting-started/examples/icedelivery.md rename to doc/website/tutorials/icedelivery.md diff --git a/doc/website/getting-started/examples/icedelivery_in_c.md b/doc/website/tutorials/icedelivery_in_c.md similarity index 100% rename from doc/website/getting-started/examples/icedelivery_in_c.md rename to doc/website/tutorials/icedelivery_in_c.md diff --git a/doc/website/getting-started/examples/icedocker.md b/doc/website/tutorials/icedocker.md similarity index 100% rename from doc/website/getting-started/examples/icedocker.md rename to doc/website/tutorials/icedocker.md diff --git a/doc/website/getting-started/examples/iceensemble.md b/doc/website/tutorials/iceensemble.md similarity index 100% rename from doc/website/getting-started/examples/iceensemble.md rename to doc/website/tutorials/iceensemble.md diff --git a/doc/website/getting-started/examples/icehello.md b/doc/website/tutorials/icehello.md similarity index 100% rename from doc/website/getting-started/examples/icehello.md rename to doc/website/tutorials/icehello.md diff --git a/doc/website/getting-started/examples/iceoptions.md b/doc/website/tutorials/iceoptions.md similarity index 100% rename from doc/website/getting-started/examples/iceoptions.md rename to doc/website/tutorials/iceoptions.md diff --git a/doc/website/getting-started/examples/iceperf.md b/doc/website/tutorials/iceperf.md similarity index 100% rename from doc/website/getting-started/examples/iceperf.md rename to doc/website/tutorials/iceperf.md diff --git a/doc/website/getting-started/examples/index.md b/doc/website/tutorials/index.md similarity index 100% rename from doc/website/getting-started/examples/index.md rename to doc/website/tutorials/index.md diff --git a/doc/website/getting-started/examples/singleprocess.md b/doc/website/tutorials/singleprocess.md similarity index 100% rename from doc/website/getting-started/examples/singleprocess.md rename to doc/website/tutorials/singleprocess.md diff --git a/doc/website/getting-started/examples/user_header.md b/doc/website/tutorials/user_header.md similarity index 100% rename from doc/website/getting-started/examples/user_header.md rename to doc/website/tutorials/user_header.md diff --git a/doc/website/getting-started/examples/waitset.md b/doc/website/tutorials/waitset.md similarity index 100% rename from doc/website/getting-started/examples/waitset.md rename to doc/website/tutorials/waitset.md diff --git a/doc/website/getting-started/examples/waitset_in_c.md b/doc/website/tutorials/waitset_in_c.md similarity index 100% rename from doc/website/getting-started/examples/waitset_in_c.md rename to doc/website/tutorials/waitset_in_c.md From 0283522bc1a38eaaaa2f4566120a2687f3c1a518 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Wed, 16 Feb 2022 16:50:30 +0100 Subject: [PATCH 02/12] iox-#743 Update used tools version and add GIF Signed-off-by: Simon Hoinkis --- doc/website/getting-started/overview.md | 2 +- tools/website/export-docu-to-website.sh | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/website/getting-started/overview.md b/doc/website/getting-started/overview.md index 7c10c0b33c..3c475cd159 100644 --- a/doc/website/getting-started/overview.md +++ b/doc/website/getting-started/overview.md @@ -117,7 +117,7 @@ Afterwards, we can start the applications which immediately connect to the RouDi When the application terminates, the runtime cleans up all resources needed for communication with RouDi. This includes all memory chunks used for the data transmission which may still be held by the application. - +![Overview](https://user-images.githubusercontent.com/8661268/74612998-b962bc80-510a-11ea-97f0-62f41c5d287b.gif) We now briefly define the main entities of an iceoryx system which were partially already used in the example above. diff --git a/tools/website/export-docu-to-website.sh b/tools/website/export-docu-to-website.sh index 8cc1dca89d..50c5d3fc87 100755 --- a/tools/website/export-docu-to-website.sh +++ b/tools/website/export-docu-to-website.sh @@ -17,12 +17,12 @@ # SPDX-License-Identifier: Apache-2.0 # Necessary tools: -# mkdocs-awesome-pages-plugin, v2.5.0 -# mkdocs-material, v7.0.7+insiders.2.5.0 +# mkdocs-awesome-pages-plugin, v2.6.0 +# mkdocs-material, v8.1.3-insiders-4.5.0q # markdown-include, v0.6.0 -# mkdocs, v1.1.2 -# Doxygen, 1.8.17 -# doxybook2, v1.3.1 +# mkdocs, v1.2.3 +# Doxygen, v1.9.1 +# doxybook2, v1.3.3 # mike, v1.0.0 set -e From baaba1b80b5fe2fa757a77ef34801adb0189e873 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Wed, 16 Feb 2022 17:11:44 +0100 Subject: [PATCH 03/12] iox-#743 Add concepts, releases notes and design Signed-off-by: Simon Hoinkis --- doc/website/.pages | 3 +++ doc/website/concepts/architecture.md | 1 + doc/website/design/service-discovery.md | 5 +++++ doc/website/release-notes/releases-notes.md | 5 +++++ 4 files changed, 14 insertions(+) create mode 100644 doc/website/concepts/architecture.md create mode 100644 doc/website/design/service-discovery.md create mode 100644 doc/website/release-notes/releases-notes.md diff --git a/doc/website/.pages b/doc/website/.pages index 3d52d8d0c1..8c93e6660c 100644 --- a/doc/website/.pages +++ b/doc/website/.pages @@ -1,7 +1,10 @@ nav: - index.md - getting-started + - concepts - tutorials + - design - FAQ.md - advanced + - Release notes - API-reference diff --git a/doc/website/concepts/architecture.md b/doc/website/concepts/architecture.md new file mode 100644 index 0000000000..c7d8fc7341 --- /dev/null +++ b/doc/website/concepts/architecture.md @@ -0,0 +1 @@ +# Eclipse iceoryx architecture diff --git a/doc/website/design/service-discovery.md b/doc/website/design/service-discovery.md new file mode 100644 index 0000000000..7614ea079c --- /dev/null +++ b/doc/website/design/service-discovery.md @@ -0,0 +1,5 @@ +--- +title: Service discovery +--- + +{! ../iceoryx/doc/design/service-discovery.md !} diff --git a/doc/website/release-notes/releases-notes.md b/doc/website/release-notes/releases-notes.md new file mode 100644 index 0000000000..28c59db607 --- /dev/null +++ b/doc/website/release-notes/releases-notes.md @@ -0,0 +1,5 @@ +--- +title: Changelog +--- + +{! ../iceoryx/CHANGELOG.md !} From 0eb2f999437ba6eebadcdb99d21c0eaedbabd5ff Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Wed, 16 Feb 2022 17:44:52 +0100 Subject: [PATCH 04/12] iox-#743 Fix MkDocs issues Signed-off-by: Simon Hoinkis --- doc/website/.pages | 2 +- doc/website/getting-started/.pages | 1 - doc/website/release-notes/{releases-notes.md => change-log.md} | 0 3 files changed, 1 insertion(+), 2 deletions(-) rename doc/website/release-notes/{releases-notes.md => change-log.md} (100%) diff --git a/doc/website/.pages b/doc/website/.pages index 8c93e6660c..33b758f7fe 100644 --- a/doc/website/.pages +++ b/doc/website/.pages @@ -6,5 +6,5 @@ nav: - design - FAQ.md - advanced - - Release notes + - release-notes - API-reference diff --git a/doc/website/getting-started/.pages b/doc/website/getting-started/.pages index 6c26ce6be8..76d6ad9dd9 100644 --- a/doc/website/getting-started/.pages +++ b/doc/website/getting-started/.pages @@ -2,4 +2,3 @@ nav: - what-is-iceoryx.md - overview.md - installation.md - - examples diff --git a/doc/website/release-notes/releases-notes.md b/doc/website/release-notes/change-log.md similarity index 100% rename from doc/website/release-notes/releases-notes.md rename to doc/website/release-notes/change-log.md From be4a46355f4a80860b875e4495d6cba0dbcb7425 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Wed, 16 Feb 2022 18:47:08 +0100 Subject: [PATCH 05/12] iox-#743 Update custom footer to latest MkDocs-Material version Signed-off-by: Simon Hoinkis --- doc/website/overrides/partials/footer.html | 22 ++++++++-------------- doc/website/stylesheets/extra.css | 2 +- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/doc/website/overrides/partials/footer.html b/doc/website/overrides/partials/footer.html index bb45cf87ed..df5ad7c67d 100644 --- a/doc/website/overrides/partials/footer.html +++ b/doc/website/overrides/partials/footer.html @@ -38,7 +38,7 @@ {% endif %}
-
+
Made with Material for MkDocs @@ -46,7 +46,7 @@
-
+
Copyright © 2021 Eclipse Foundation
Privacy policy
Terms of use
@@ -57,7 +57,7 @@
-
+
Sponsored by

@@ -68,27 +68,21 @@
-
+
+ class="md-social__link"> {% include ".icons/material/github.svg" %} -
-
+ class="md-social__link"> {% include ".icons/material/chat.svg" %} -
-
+ target="_blank" rel="noopener" title="Developer Meetup" class="md-social__link"> {% include ".icons/material/calendar.svg" %} -
-
+ title="Mailing List" class="md-social__link"> {% include ".icons/material/email-multiple.svg" %}
diff --git a/doc/website/stylesheets/extra.css b/doc/website/stylesheets/extra.css index d38d504913..8e18126053 100644 --- a/doc/website/stylesheets/extra.css +++ b/doc/website/stylesheets/extra.css @@ -71,7 +71,7 @@ strong { justify-content: left; } -.md-footer-copyright { +.md-copyright { margin: 0.6rem; } From 476925d46dce3e84403a36cc97e127aa62c31e32 Mon Sep 17 00:00:00 2001 From: Dietrich Kroenke Date: Thu, 17 Feb 2022 10:40:31 +0100 Subject: [PATCH 06/12] iox-#743 Restructure changelogs for website --- CHANGELOG.md | 659 ------------------ doc/website/release-notes/.pages | 6 + doc/website/release-notes/change-log.md | 5 - .../release-notes/iceoryx-unreleased.md | 27 + doc/website/release-notes/iceoryx-v1-0-0.md | 297 ++++++++ doc/website/release-notes/iceoryx-v1-0-1.md | 19 + doc/website/release-notes/iceoryx-v1-0-2.md | 9 + doc/website/release-notes/iceoryx-v2-0-0.md | 346 +++++++++ 8 files changed, 704 insertions(+), 664 deletions(-) delete mode 100644 CHANGELOG.md create mode 100644 doc/website/release-notes/.pages delete mode 100644 doc/website/release-notes/change-log.md create mode 100644 doc/website/release-notes/iceoryx-unreleased.md create mode 100644 doc/website/release-notes/iceoryx-v1-0-0.md create mode 100644 doc/website/release-notes/iceoryx-v1-0-1.md create mode 100644 doc/website/release-notes/iceoryx-v1-0-2.md create mode 100644 doc/website/release-notes/iceoryx-v2-0-0.md diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index eae99beb47..0000000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,659 +0,0 @@ -# Change Log - -## [vX.X.X](https://github.com/eclipse-iceoryx/iceoryx/tree/v1.0.0) (unreleased) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v1.0.1...master) - -**Features:** - -- Partial enable iceoryx building with msvc2015+clang [\#965](https://github.com/eclipse-iceoryx/iceoryx/issues/965) -- C binding for posh configuration [\#930](https://github.com/eclipse-iceoryx/iceoryx/issues/930) -- Enhance MacOS performance with timed{send,receive} functionality in unix domain socket[\#903](https://github.com/eclipse-iceoryx/iceoryx/issues/903) -- Multi-Publisher support for DDS gateway and generic gateway class [\#900](https://github.com/eclipse-iceoryx/iceoryx/issues/900) -- Replace `iox-gw-iceoryx2dds` and `iox-gw-dds2iceoryx` gateways with `iox-dds-gateway` [\#900](https://github.com/eclipse-iceoryx/iceoryx/issues/900) -- Enhance posixCall[\#805](https://github.com/eclipse-iceoryx/iceoryx/issues/805) -- New chunk available callback for the new C++[\#391](https://github.com/eclipse-iceoryx/iceoryx/issues/391) -- Git Hooks on iceoryx[\#486](https://github.com/eclipse-iceoryx/iceoryx/issues/486) -- static memory alternative for std::function[\#391](https://github.com/eclipse-iceoryx/iceoryx/issues/391) -- Adding support for Helix QAC 2021.1[\#755](https://github.com/eclipse-iceoryx/iceoryx/issues/755) thanks to @toniglandy1 -- Axivion analysis on CI[\#409](https://github.com/eclipse-iceoryx/iceoryx/issues/409) -- Cpptoml can be provided by an external source[\#951](https://github.com/eclipse-iceoryx/iceoryx/issues/) -- Extend cxx::optional constructor for in place construction so that copy/move for values inside the optional even could be deleted[\#967](https://github.com/eclipse-iceoryx/iceoryx/issues/967) -- Add templated `from`/`into` free functions to formalize conversions from enums and other types [#992](https://github.com/eclipse-iceoryx/iceoryx/issues/992) -- UniqueId class for unique IDs within a process [#1010](https://github.com/eclipse-iceoryx/iceoryx/issues/1010) -- Add requirePublisherHistorySupport option at subscriber side (if set to true requires historyRequest <= historyCapacity to be eligible for connection) [#1029](https://github.com/eclipse-iceoryx/iceoryx/issues/1029) -- Add `/tools/scripts/ice_env.sh` shell script to provide simple access to docker containers for CI debugging [#1049](https://github.com/eclipse-iceoryx/iceoryx/issues/1049) -- Introduce `cxx::FunctionalInterface` to enrich nullable classes with `and_then`, `or_else`, `value_or`, `expect` [\#996](https://github.com/eclipse-iceoryx/iceoryx/issues/996) -- Add C++17 `std::perms` as `cxx::perms` to `iceoryx_hoofs/cxx/filesystem.hpp`. [#1059](https://github.com/eclipse-iceoryx/iceoryx/issues/1059) -- Support FreeBSD as a representative for the UNIX platforms [#1054](https://github.com/eclipse-iceoryx/iceoryx/issues/1054) -- Add event parameter to `findService` method [#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) -- Implement stream operator for `ChunkReceiveResult` and `AllocationError` to be able to use it with ostream and LogStream [\#1062](https://github.com/eclipse-iceoryx/iceoryx/issues/1062) -- Replace IPC-channel-based `findService` with pub/sub-based on [#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) - -**Bugfixes:** - -- Fix support for libc++ on clang[\#905](https://github.com/eclipse-iceoryx/iceoryx/issues/905) -- Fix warnings for gcc-11.1[\#838](https://github.com/eclipse-iceoryx/iceoryx/issues/838) -- Incremental builds with the build script are broken[\#821](https://github.com/eclipse-iceoryx/iceoryx/issues/821) -- Compile failed because of missing for GCC 11[\#811](https://github.com/eclipse-iceoryx/iceoryx/issues/811) thanks to @homalozoa -- Unable to build cyclone dds idlpp-cxx [\#736](https://github.com/eclipse-iceoryx/iceoryx/issues/736) -- Fix format string issues with introspection client [\#960](https://github.com/eclipse-iceoryx/iceoryx/issues/960) thanks to @roehling -- Add support for Multi-Arch install destinations [\#961](https://github.com/eclipse-iceoryx/iceoryx/issues/961) thanks to @roehling -- Fix a few misspellings in log messages [\#962](https://github.com/eclipse-iceoryx/iceoryx/issues/962) thanks to @roehling -- Fix typos in goals/non-goals document [\#968](https://github.com/eclipse-iceoryx/iceoryx/issues/968) -- Catch deserialization errors for enums in publisher and subscriber options [\#989](https://github.com/eclipse-iceoryx/iceoryx/issues/989) -- Fix linker error on QNX [\#1013](https://github.com/eclipse-iceoryx/iceoryx/issues/1013) -- When posix mutex fails a correct error message is reported on the console [\#999](https://github.com/eclipse-iceoryx/iceoryx/issues/999) -- Only use `std::result_of` for C++14 to be able to use iceoryx in C++20 projects [\#1076](https://github.com/eclipse-iceoryx/iceoryx/issues/1076) -- Set stack size for windows in singleprocess example and posh tests [\#1082](https://github.com/eclipse-iceoryx/iceoryx/issues/1082) - -**Refactoring:** - -- Handle nullptr callbacks in waitset and listener[\#932](https://github.com/eclipse-iceoryx/iceoryx/issues/932) -- Add clang-tidy rules for iceoryx_hoofs[\#889](https://github.com/eclipse-iceoryx/iceoryx/issues/889) -- Move all tests into an anonymous namespace[\#563](https://github.com/eclipse-iceoryx/iceoryx/issues/563) -- Refactor smart_c to use contract by design and expected[\#418](https://github.com/eclipse-iceoryx/iceoryx/issues/418) -- PoshRuntime Mock[\#449](https://github.com/eclipse-iceoryx/iceoryx/issues/449) -- Clean-up Doxygen for all methods in headers[\#583](https://github.com/eclipse-iceoryx/iceoryx/issues/583) -- Iox-#590 Rename utils to hoofs[\#790](https://github.com/eclipse-iceoryx/iceoryx/pull/790) -- Iox-#590 plantuml in design documentation[\#787](https://github.com/eclipse-iceoryx/iceoryx/pull/787) -- Refine quality levels[\#425](https://github.com/eclipse-iceoryx/iceoryx/issues/425) -- Clean-up std::terminate usage[\#261](https://github.com/eclipse-iceoryx/iceoryx/issues/261) -- Add Quality Declaration Document[\#910](https://github.com/eclipse-iceoryx/iceoryx/issues/910) -- Make `cxx::string::capacity` a `static` method[\#979](https://github.com/eclipse-iceoryx/iceoryx/issues/979) -- Restructure iceoryx tools [\#471](https://github.com/eclipse-iceoryx/iceoryx/issues/471) -- Use cxx::expected for MemoryManager::getChunk [\#954](https://github.com/eclipse-iceoryx/iceoryx/pull/991) -- Upgrade GTest/GMock to v1.10 [\#841](https://github.com/eclipse-iceoryx/iceoryx/issues/841) -- Remove the requirement for INVALID_STATE for the cxx::expected [\#987](https://github.com/eclipse-iceoryx/iceoryx/issues/987) -- Add unique test identifers [\#988](https://github.com/eclipse-iceoryx/iceoryx/issues/988) -- Remove `ApplicationPort` and `ApplicationPortData` classes [\#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) -- Remove creation pattern from `MemoryMap` and replace it with `MemoryMapBuilder` [\#1036](https://github.com/eclipse-iceoryx/iceoryx/issues/1036) -- Fix error handling of `TypedUniqueId` and refactor it to `UniquePortId` [\#861](https://github.com/eclipse-iceoryx/iceoryx/issues/861) -- Updating Codecov API and enforce CMake version 3.16 for building iceoryx [\#774](https://github.com/eclipse-iceoryx/iceoryx/issues/774) and [\#1031](https://github.com/eclipse-iceoryx/iceoryx/issues/1031) -- Remove `InvalidIdString` and `isValid()` from `ServiceDescription`, replace Wildcard string with `iox::cxx::nullopt` [\#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) -- Remove creation pattern from `SharedMemory` and replace it with `SharedMemoryBuilder` [\#1036](https://github.com/eclipse-iceoryx/iceoryx/issues/1036) -- Remove the leading slash requirement from the name of a shared memory in `SharedMemory` and `SharedMemoryObject` [\#439](https://github.com/eclipse-iceoryx/iceoryx/issues/439) - -**New API features:** - -- Introduce `iceoryx_hoofs/cxx/filesystem.hpp` which implements `std::perms` as `cxx::perms`. - -```cpp -#include "iceoryx_hoofs/cxx/filesystem.hpp" - -// ... -cxx::perms filePermissions; -filePermissions = cxx::perms::owner_read | cxx::perms::group_write; -std::cout << filePermissions << std::endl; -``` - -**API Breaking Changes:** - -The CMake files in iceoryx expect to have CMake version 3.16 or greater installed, otherwise the build fails. -(Hint: Ubuntu 18 users can install `cmake-mozilla` from the universe repository provided by Canonical) - -Dependency for building the iceoryx_dds gateway changed from `openjdk` (Java) to `bison` - -Change include from `iceoryx_hoofs/cxx/helplets.hpp` to `iceoryx_hoofs/cxx/requires.hpp` -when using `cxx::Expects` or `cxx::Ensures` -```cpp -// before -#include "iceoryx_hoofs/cxx/helplets.hpp" - -iox::cxx::Expects(someCondition); -iox::cxx::Ensures(anotherCondition); - -// after -#include "iceoryx_hoofs/cxx/requires.hpp" - -iox::cxx::Expects(someCondition); -iox::cxx::Ensures(anotherCondition); -``` - - -Replace Creation pattern from `MemoryMap` with `MemoryMapBuilder`. - -```cpp -// before -auto memoryMapResult = posix::MemoryMap::create(baseAddress, length, fileDescriptor, accessMode, flags, offset); - -// after -auto memoryMapResult = posix::MemoryMapBuilder().baseAddressHint(baseAddress) - .length(length).fileDescriptor(fileDescriptor) - .accessMode(accessMode).flags(flags) - .offset(0).create(); -``` - -Rename utils to hoofs: - -- in CMake you need now to find and link the package `iceoryx_hoofs` instead of `iceoryx_utils` - -```cmake -# before -find_package(iceoryx_utils REQUIRED) -target_link_libraries(${target} - iceoryx_utils::iceoryx_utils) - -# after -find_package(iceoryx_hoofs REQUIRED) -target_link_libraries(${target} - iceoryx_hoofs::iceoryx_hoofs) -``` - -- the include paths for `iceoryx_utils` are now `iceoryx_hoofs` - -```cpp -// before -#include "iceoryx_utils/cxx/string.hpp" - -// after -#include "iceoryx_hoofs/cxx/string.hpp" -``` - -Refactoring SmartC: - -- Renaming SmartC wrapper to posixCall. -- Removed `getErrorString()` from posixCall, please use `getHumanReadableErrnum()` instead. -- Enhanced posixCall to handle a common case were multiple errnos are ignored just to suppress error logging - -```cpp -// before -#include "iceoryx_utils/cxx/smart_c.hpp" - -auto unlinkCallPublisher = iox::cxx::makeSmartC( - unlink, iox::cxx::ReturnMode::PRE_DEFINED_ERROR_CODE, {ERROR_CODE}, {ENOENT}, sockAddrPublisher.sun_path); - - if (unlinkCallPublisher.hasErrors()) - { - std::cout << "unlink error" << std::endl; - exit(1); - } - -// after -#include "iceoryx_utils/posix_wrapper/posix_call.hpp" - -iox::posix::posixCall(unlink)(sockAddrPublisher.sun_path) - .failureReturnValue(ERROR_CODE) - .ignoreErrnos(ENOENT, EBUSY) // can be a comma-separated list of errnos - .evaluate() - .or_else([](auto& r) { - std::cout << "unlink error " << r.getHumanReadableErrnum() << std::endl; - exit(1); - }); -``` - -A `ServiceDescription` is now only string-based and no more wildcards are allowed. -A well-defined `ServiceDescription` consists of three non-empty strings. - -```cpp -// before -ServiceDescription myServiceDescription1(1U, 2U, 3U); -ServiceDescription myServiceDescription3("First", "Second"); -ServiceDescription myServiceDescription3(iox::capro::AnyServiceString, iox::capro::AnyInstanceString, iox::capro::AnyEventString); - -// after -ServiceDescription myServiceDescription1("Foo", "Bar", "Baz"); -ServiceDescription myServiceDescription2("First", "Second", "DontCare"); -ServiceDescription myServiceDescription3("Foo", "Bar", "Baz"); -``` - -The following classes have now an constructor marked as `explicit`: - -```cpp -explicit DeadlineTimer(const iox::units::Duration timeToWait); -explicit GenericRAII(const std::function& cleanupFunction); -explicit mutex(const bool f_isRecursive); -explicit PosixUser(const uid_t f_id); -explicit PosixUser(const string_t& f_name); -``` - -Renaming in `FileReader` class and logging of iceoryx_hoofs - -```cpp -// before -iox::cxx::FileReader reader("filename"); -std::string str; -if(reader.IsOpen()) { - reader.ReadLine(str); -} - -static auto& logger = CreateLogger("", "", iox::log::LogManager::GetLogManager().DefaultLogLevel()); - -// after -iox::cxx::FileReader reader("filename"); -std::string str; -if(reader.isOpen()) { - reader.readLine(str); -} - -static auto& logger = createLogger("", "", iox::log::LogManager::GetLogManager().DefaultLogLevel()); -``` - -Instead of using a constructor a `ServiceDescription` is now deserialized via a -static method with error handling: - -```cpp -// before -iox::cxx::Serialization serializedObj; -iox::capro::ServiceDescription service(serializedObj); - -// after -iox::cxx::Serialization serialisedObj; -capro::ServiceDescription::deserialize(serialisedObj) - .and_then([](auto& value){ - // Do something with the deserialized object - }) - .or_else([](auto& error){ - // Handle the error - }); -``` - -The service-related methods have been moved from `PoshRuntime` to `ServiceDiscovery`. The `offerService` -and `stopOfferService` methods have been removed and `findService` has now an additional event paramter: - -```cpp -// before -#include "iceoryx_posh/runtime/posh_runtime.hpp" - -poshRuntime.offerService(myServiceDescription); -poshRuntime.stopOfferService(myServiceDescription); -poshRuntime.findService({"ServiceA", iox::capro::AnyInstanceString}); - -// after -#include "iceoryx_posh/runtime/service_discovery.hpp" - -serviceDiscovery.findService("ServiceA", Wildcard, Wildcard); -``` - -The `iox::cxx::expected` has dropped the requirement for `INVALID_STATE`. With this, the -`ErrorTypeAdapter` which was necessary for non enum types was also removed. The specialization -of `ErrorTypeAdapter` for custom types must therefore also be removed in the user code. - -The `InvalidIdString` was removed from `ServiceDescription` and the Wildcard string was replaced -with a `iox::cxx::nullopt`. With this, every string is allowed within the `ServiceDescription`. -The default `ServiceDescription` consists of empty strings. - -- The queue port policy enums are adjusted to use them with `Client` and `Server`. - -The `QueueFullPolicy::BLOCK_PUBLISHER` is replaced with the more generic `QueueFullPolicy::BLOCK_PRODUCER`. - -```cpp -// old -iox::popo::SubscriberOptions options; -options.queueFullPolicy = iox::popo::QueueFullPolicy::BLOCK_PUBLISHER; - -// new -iox::popo::SubscriberOptions options; -options.queueFullPolicy = iox::popo::QueueFullPolicy::BLOCK_PRODUCER; -``` - -Similar, for `binding_c` it is `QueueFullPolicy_BLOCK_PRODUCER` instead of `QueueFullPolicy_BLOCK_PUBLISHER` - -```c -// old -iox_sub_options_t options; -iox_sub_options_init(&options); -options.queueFullPolicy = QueueFullPolicy_BLOCK_PUBLISHER; - -// new -iox_sub_options_t options; -iox_sub_options_init(&options); -options.queueFullPolicy = QueueFullPolicy_BLOCK_PRODUCER; -``` - -The `SubscriberTooSlowPolicy` is replaced with the more generic `ConsumerTooSlowPolicy` and -`SubscriberTooSlowPolicy::WAIT_FOR_SUBSCRIBER` became `ConsumerTooSlowPolicy::WAIT_FOR_CONSUMER`. - -```cpp -// old -iox::popo::PublisherOptions options; -options.subscriberTooSlowPolicy = iox::popo::SubscriberTooSlowPolicy::WAIT_FOR_SUBSCRIBER; - -// new -iox::popo::PublisherOptions options; -options.subscriberTooSlowPolicy = iox::popo::ConsumerTooSlowPolicy::WAIT_FOR_CONSUMER; -``` - -And with the `binding_c` - -```c -// old -iox_pub_options_t options; -iox_pub_options_init(&options); -options.subscriberTooSlowPolicy = SubscriberTooSlowPolicy_WAIT_FOR_SUBSCRIBER; - -// new -iox_pub_options_t options; -iox_pub_options_init(&options); -options.subscriberTooSlowPolicy = ConsumerTooSlowPolicy_WAIT_FOR_CONSUMER; -``` - -- The `CaproMessageSubType` enum is renamed to `CaproServiceType` and the values are renamed from `NOSUBTYPE`, - `SERVICE`, `EVENT` and `FIELD` to `NONE`, `PUBLISHER` and `SERVER` - -This change only affects `InterfacePorts` which used this enum to communicate whether the `CaproMessage` -was from a `SERVICE`, `EVENT` or `FIELD`. This was quite ara::com specific and with the introduction of the `ServerPort` -changes were needed. The distinction between a `FIELD` and an `EVENT` can be made by checking -`CaproMessage::m_historyCapacity`. - -```cpp -// old -caproMessage.m_subType = CaproMessageSubType::EVENT; - -// new -caproMessage.m_serviceType = CaproServiceType::PUBLISHER; -``` - -## [v1.0.1](https://github.com/eclipse-iceoryx/iceoryx/tree/v1.0.0) (2021-06-15) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v1.0.0...v1.0.1) - -**Description:** -This is the first bugfix release for Eclipse iceoryx 1.0.0. We made minor changes in the documentation and added several patches. - -Compared to the feature content of the release 1.0.0, the following bug tickets where resolved: - -**Bugfixes:** - -- Github CI Integrationtests failing[\#825](https://github.com/eclipse-iceoryx/iceoryx/issues/825) -- Segmentation Fault in FileLock [\#808](https://github.com/eclipse-iceoryx/iceoryx/issues/808) -- cpptoml cannot be build due to failing examples [\#791](https://github.com/eclipse-iceoryx/iceoryx/issues/791) -- Missing template parameter for user-header in typed C++ API [\#768](https://github.com/eclipse-iceoryx/iceoryx/issues/768) -- Fix issues with generating website documentation [\#743](https://github.com/eclipse-iceoryx/iceoryx/issues/743) -- Starting an application after RouDi was terminated floods the cmd line with error messages [\#654](https://github.com/eclipse-iceoryx/iceoryx/issues/654) - -## [v1.0.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v1.0.0) (2021-04-15) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.90.7...v1.0.0) - -Description: -This is the first major release for Eclipse iceoryx. That means it is the first release with long-term support and the adopters of iceoryx can rely on a stable API. The release called Almond allows for true zero-copy inter-process-communication on Linux, QNX and MacOS and provides C and modern C++ user APIs. This release is supported until 2022-04-01. - -Compared to the feature content of the initial contribution, the main new features are: - -**Features:** - -- Introduction of a C API -- Refactoring of C++ API -- MacOS support -- Support for n:m communication -- Bridge to Eclipse Cyclone DDS - -See [Eclipse iceoryx 1.0.0 (Almond)](https://projects.eclipse.org/projects/technology.iceoryx/releases/1.0.0-almond) for more information. - -- Reserved chunkinfo user payload header [\#14](https://github.com/eclipse-iceoryx/iceoryx/issues/14) -- New chunk available callback for the new C++ and C APIs [\#350](https://github.com/eclipse-iceoryx/iceoryx/issues/350) -- Add generic QNX toolchain files [\#609](https://github.com/eclipse-iceoryx/iceoryx/issues/609) -- Introduction of Runnables/Nodes for the new APIs [\#349](https://github.com/eclipse-iceoryx/iceoryx/issues/349) - -**Refactoring:** - -- Added tests to iceoryx_posh and iceoryx_utils [\#496](https://github.com/eclipse-iceoryx/iceoryx/issues/496), - [\#484](https://github.com/eclipse-iceoryx/iceoryx/issues/484), [\#454](https://github.com/eclipse-iceoryx/iceoryx/issues/454) -, [\#240](https://github.com/eclipse-iceoryx/iceoryx/issues/240) -- Add [[nodiscard]] keyword to cxx::expected class [\#624](https://github.com/eclipse-iceoryx/iceoryx/issues/624) -- Refactor Relocatable Pointer [\#605](https://github.com/eclipse-iceoryx/iceoryx/issues/605) -- Integration test with RouDi and new API elements [\#378](https://github.com/eclipse-iceoryx/iceoryx/issues/378) -- Refactoring copyright headers [\#483](https://github.com/eclipse-iceoryx/iceoryx/issues/483) -- Integration of Unix Domain Sockets [\#381](https://github.com/eclipse-iceoryx/iceoryx/issues/381) -- Add how to install after build to guide [\#533](https://github.com/eclipse-iceoryx/iceoryx/issues/533) -- Add maven and openjdk-14-jdk-headless as build prerequisites [\#525](https://github.com/eclipse-iceoryx/iceoryx/issues/525) -- Refine modern pub/sub API [\#408](https://github.com/eclipse-iceoryx/iceoryx/issues/408) -- Replace introspection threads with PeriodicTask [\#489](https://github.com/eclipse-iceoryx/iceoryx/issues/489) -- Enable UndefinedBehaviorSanitizer [\#489](https://github.com/eclipse-iceoryx/iceoryx/issues/459) -- Move iceoryx from eclipse to iceoryx-eclipse organization [\#467](https://github.com/eclipse-iceoryx/iceoryx/issues/467) - -**Fixed bugs:** - -- TOML parser exceptions will crash RouDi [\#622](https://github.com/eclipse-iceoryx/iceoryx/issues/622) -- fail build on ubuntu 16.04 with g++ 5.4.0 [\#495](https://github.com/eclipse-iceoryx/iceoryx/issues/495) -- Wrong handling of applications that are started multiple times in parallel [\#404](https://github.com/eclipse-iceoryx/iceoryx/issues/404) -- Expected and_then/or_else calling callable without checking [\#599](https://github.com/eclipse-iceoryx/iceoryx/issues/599) -- std::void_t used in code [\#591](https://github.com/eclipse-iceoryx/iceoryx/issues/591) -- Fix resource leak caused by move ctor/assignment [\#542](https://github.com/eclipse-iceoryx/iceoryx/issues/542) -- nanoseconds in units::Duration are truncated [\#190](https://github.com/eclipse-iceoryx/iceoryx/issues/190) -- popo/user_trigger.hpp:33:5: error: exception specification of explicitly defaulted default constructor does not match the calculated one [\#494](https://github.com/eclipse-iceoryx/iceoryx/issues/494) -- iceoryx libs should be build as static libs [\#509](https://github.com/eclipse-iceoryx/iceoryx/issues/509) -- Build with gcov is broken [\#497](https://github.com/eclipse-iceoryx/iceoryx/issues/497) - -## [v0.99.7](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.7) (2021-04-09) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.6...v0.99.7) - -- cleanup testing libraries; if used outside of iceoryx, include paths and link targets must be adjusted -- build iceoryx_utils, iceoryx_posh and iceoryx_binding_c as shared libraries in colcon - -## [v0.99.6](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.6) (2021-04-08) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.5...v0.99.6) - -- Add git to dependency list in iceoryx_posh for RHEL - -## [v0.99.5](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.5) (2021-04-08) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.4...v0.99.5) - -- Add libatomic to dependency list in iceoryx_utils for RHEL - -## [v0.99.4](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.4) (2021-04-06) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.3...v0.99.4) - -- Increase version number to trigger build again on ROS buildfarm - -## [v0.99.3](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.3) (2021-04-06) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.2...v0.99.3) - -- Increase version number to trigger build again on ROS buildfarm - -## [v0.99.2](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.2) (2021-04-03) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.0...v0.99.2) - -- Prepare package release for iceoryx 1.0 [\#670](https://github.com/eclipse-iceoryx/iceoryx/issues/670) - -## [v0.99.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.0) (2021-04-01) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.90.0...v0.99.0) - -Intermediate Milestone before the final 1.0.0 release, see v.1.0.0 above for the changes. - -## [v0.90.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.90.0) (2020-12-22) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.17.0...v0.90.0) - -Pre-Release for new and stable APIs and n:m pub/sub communication - -This is a pre-release for our first long-term-support release that is coming soon (will be iceoryx_1.0.0). - -We had a major refactoring of the iceoryx communication infrastructure which allows to do n:m communication now and provides a flexible history functionality for late joining subscribers. - -This new infrastructure is also made to support request/response communication as a next step [(\#27)](https://github.com/eclipse-iceoryx/iceoryx/issues/27). The C++ API has been completely reworked and a new C API has been added. - -We will give the API draft some weeks for finalizing the features and to incorporate feedback from the community. There's an issue in which we collect and discuss the final modifications [(\#408)](https://github.com/eclipse-iceoryx/iceoryx/issues/408). - -**Features:** - -- Complete refactoring of publishers and subscribers from shared memory data structures to user APIs, [\#252](https://github.com/eclipse-iceoryx/iceoryx/issues/252) -- First versions of new APIs for C++ and C [\#252](https://github.com/eclipse-iceoryx/iceoryx/issues/252) -- n:m publish/subscribe communication now possible [\#25](https://github.com/eclipse-iceoryx/iceoryx/issues/25) -- First version of a DDS gateway. Cyclone DDS already integrated, FastDDS on it's way [\#64](https://github.com/eclipse-iceoryx/iceoryx/issues/64), [\#65](https://github.com/eclipse-iceoryx/iceoryx/issues/65) -- New github actions for Mac OS and colcon build [\#175](https://github.com/eclipse-iceoryx/iceoryx/issues/175), [\#276](https://github.com/eclipse-iceoryx/iceoryx/issues/276), [\#328](https://github.com/eclipse-iceoryx/iceoryx/issues/328) -- Adjustable capacity for the lockfree queue [\#216](https://github.com/eclipse-iceoryx/iceoryx/issues/216) -- Check the files have a copyright header [\#346](https://github.com/eclipse-iceoryx/iceoryx/issues/346) - -**Refactoring:** - -- Refactoring of waitset [\#341](https://github.com/eclipse-iceoryx/iceoryx/issues/341) -- create multi publisher example [\#394](https://github.com/eclipse-iceoryx/iceoryx/issues/394) -- Full IceOryx Public API Cheat Sheet [\#283](https://github.com/eclipse-iceoryx/iceoryx/issues/283) -- Rework build and test steps in iceoryx [\#433](https://github.com/eclipse-iceoryx/iceoryx/issues/433) -- Extend iceperf example to C API [\#453](https://github.com/eclipse-iceoryx/iceoryx/issues/453) -- Remove default parameter from PoshRuntime::getInstance() [\#382](https://github.com/eclipse-iceoryx/iceoryx/issues/382) -- Enable Sanitizer in Debug Build and Unit Tests [\#141](https://github.com/eclipse-iceoryx/iceoryx/issues/141) -- Minor RouDi cleanups [\#91](https://github.com/eclipse-iceoryx/iceoryx/issues/91) -- C++14 [\#220](https://github.com/eclipse-iceoryx/iceoryx/issues/220) -- Replace occurence of std::list by cxx::list [\#221](https://github.com/eclipse-iceoryx/iceoryx/issues/221) - -**Fixed bugs:** - -- SegFault in iox-roudi on startup [\#447](https://github.com/eclipse-iceoryx/iceoryx/issues/447) -- Mocks can cause segfaults/undefined behavior [\#427](https://github.com/eclipse-iceoryx/iceoryx/issues/427) -- Chunks are lost forever when having an overflow in a variant queue of type FIFO [\#456](https://github.com/eclipse-iceoryx/iceoryx/issues/456) -- RouDi does not free shared memory properly on x86 Linux [\#324](https://github.com/eclipse-iceoryx/iceoryx/issues/324) -- WaitSet behavior wrong in `waitAndReturnFulfilledConditions` [\#388](https://github.com/eclipse-iceoryx/iceoryx/issues/388) -- Incorrect usage of strncpy [\#374](https://github.com/eclipse-iceoryx/iceoryx/issues/374) -- Global Instantiation of Publisher/Subscriber created core dump [\#327](https://github.com/eclipse-iceoryx/iceoryx/issues/327) - -## [v0.17.0](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0) (2020-08-27) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC6...v0.17.0)\ -MacOS support and preparations for new API - -**Packages:** - -- iceoryx_posh (Quality level 4) -- iceoryx_utils (Quality level 4) -- iceoryx_examples (Quality level 5) - -**Features:** - -- MacOS support [\#32](https://github.com/eclipse-iceoryx/iceoryx/issues/32) -- Major RouDi refactorings [\#70](https://github.com/eclipse-iceoryx/iceoryx/issues/70), [\#59](https://github.com/eclipse-iceoryx/iceoryx/issues/59), [\#78](https://github.com/eclipse-iceoryx/iceoryx/issues/78) -- Preparations for new iceoryx API [\#25](https://github.com/eclipse-iceoryx/iceoryx/issues/25) -- iceoryx to cyclonedds gateway [\#64](https://github.com/eclipse-iceoryx/iceoryx/issues/64) -- Introduce cxx::function_ref [\#86](https://github.com/eclipse-iceoryx/iceoryx/issues/86) - -**Fixed bugs:** - -- POSIX timer improvements [\#120](https://github.com/eclipse-iceoryx/iceoryx/issues/120), [\#167](https://github.com/eclipse-iceoryx/iceoryx/issues/167) -- Memory Synchronisation Issue in FiFo [\#119](https://github.com/eclipse-iceoryx/iceoryx/issues/119) -- Roudi MessageQueue thread startup [\#171](https://github.com/eclipse-iceoryx/iceoryx/issues/171) -- Destructor fix MessageQueue and UnixDomainSocket [\#150](https://github.com/eclipse-iceoryx/iceoryx/issues/150) -- RouDi ressource clean-up [\#113](https://github.com/eclipse-iceoryx/iceoryx/issues/113) - -## [v0.17.0_RC6](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC6) (2020-07-29) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC5...ICEORYX_0.17.0_RC6)\ -Release Candidate for 0.17.0 release - -**Fixed bugs:** - -- Sporadic timing test failure [\#120](https://github.com/eclipse-iceoryx/iceoryx/issues/120) -- Improvement: Merge Mempool Introspection data into one sample [\#210](https://github.com/eclipse-iceoryx/iceoryx/issues/210) - -## [v0.17.0_RC5](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC5) (2020-07-29) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC4...ICEORYX_0.17.0_RC5)\ -Release Candidate for 0.17.0 release - -**Fixed bugs:** - -- Callable can outrun periodicity of POSIX timer [\#161](https://github.com/eclipse-iceoryx/iceoryx/issues/161) - -## [v0.17.0_RC4](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC4) (2020-07-29) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC3...ICEORYX_0.17.0_RC4)\ -Release Candidate for 0.17.0 release - -**Fixed bugs:** - -- The destroy method of MessageQueue and UnixDomainSocket does not fully invalidate the object [\#150](https://github.com/eclipse-iceoryx/iceoryx/issues/150) -- Roudi message queue thread startup [\#171](https://github.com/eclipse-iceoryx/iceoryx/issues/171) - -## [v0.17.0_RC3](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC3) (2020-07-27) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC2...ICEORYX_0.17.0_RC3)\ -Release Candidate for 0.17.0 release - -**Features:** - -- cmake options for deployment parameter [\#142](https://github.com/eclipse-iceoryx/iceoryx/issues/142) - -## [v0.17.0_RC2](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC2) (2020-07-27) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC1...ICEORYX_0.17.0_RC2)\ -Release Candidate for 0.17.0 release - -**Refactoring:** - -- Refactoring of logging [\#88](https://github.com/eclipse-iceoryx/iceoryx/issues/88) -- Remove const_cast wherever possible [\#76](https://github.com/eclipse-iceoryx/iceoryx/issues/76) -- Remove asynchronous service discovery feature [\#90](https://github.com/eclipse-iceoryx/iceoryx/issues/90) -- qacpp-4.5.0-2427 [\#93](https://github.com/eclipse-iceoryx/iceoryx/issues/93) -- Usage of github actions to build pull requests [\#89](https://github.com/eclipse-iceoryx/iceoryx/issues/89) -- Reduce default memory consumption with config and mempools [\#78](https://github.com/eclipse-iceoryx/iceoryx/issues/78) - -**Fixed bugs:** - -- Sporadic timing test failure [\#120](https://github.com/eclipse-iceoryx/iceoryx/issues/120) -- Make SOFI real size equal to the specified one [\#105](https://github.com/eclipse-iceoryx/iceoryx/issues/105) -- increase padding in introspection [\#117](https://github.com/eclipse-iceoryx/iceoryx/issues/117) -- Increase the process waiting for RouDi timeout to 60 seconds [\#110](https://github.com/eclipse-iceoryx/iceoryx/issues/110) - -## [v0.17.0_RC1](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC1) (2020-03-24) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.16.1...ICEORYX_0.17.0_RC1)\ -Release Candidate for 0.17.0 release - -**Features:** - -- Memory abstraction, modularization of RouDi and fixes [\#59](https://github.com/eclipse-iceoryx/iceoryx/issues/59) -- Memory abstraction and RouDi modularization step 2, fixed string improvements and other fixes [\#70](https://github.com/eclipse-iceoryx/iceoryx/issues/70) - -**Fixed bugs:** - -- Can't compile due to missing CPPTOML [\#67](https://github.com/eclipse-iceoryx/iceoryx/issues/67) - -## [v0.16.1](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.16.1) (2020-03-02) - -[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.16.0...v0.16.1)\ -Support for rmw_iceoryx with ROS2 python API - -**Packages:** - -- POSH (POSIX Shared Memory) -- Utils -- Iceoryx examples - -**Features:** - -- Capability to cleanup shared memory resources during process lifetime [\#51](https://github.com/eclipse-iceoryx/iceoryx/issues/51) - -**Fixed bugs:** - -- compile error with clang [\#43](https://github.com/eclipse-iceoryx/iceoryx/issues/43) -- CMqInterface unit tests are failing caused by unneeded move operations [\#56](https://github.com/eclipse-iceoryx/iceoryx/issues/56) - -**Known limitations:** - -- ```RouDi --version``` shows 0.16.0.1 instead of 0.16.1 - -## [v0.16.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.16.0) (2019-12-16) - -Introspection, performance test and flexible mapping of shared memory - -**Packages:** - -- POSH (POSIX Shared Memory) -- Utils -- Iceoryx examples - -**Features:** - -- Introspection client for live debugging of iceoryx [\#21](https://github.com/eclipse-iceoryx/iceoryx/issues/21) -- Flexible mapping of shared memory in virtual address spaces [\#19](https://github.com/eclipse-iceoryx/iceoryx/issues/19) -- Performance test to measure inter-process latency [\#17](https://github.com/eclipse-iceoryx/iceoryx/issues/17) -- Docker build [\#15](https://github.com/eclipse-iceoryx/iceoryx/issues/15) (Thanks to @Mr-Slippery ) - -**Fixed bugs:** - -- Payload size not updated by allocateChunk [\#10](https://github.com/eclipse-iceoryx/iceoryx/issues/10) -- Failure in runnable creation [\#23](https://github.com/eclipse-iceoryx/iceoryx/issues/23) - -Command for generating git log for merge commits between two tags - -```bash -**git log --merges --first-parent master --pretty=format:"%h %<(10,trunc)%aN %C(white)%<(15)%aD%Creset %C(red bold)%<(15)%D%Creset %s" ... > diff_merge_commit.log** -``` diff --git a/doc/website/release-notes/.pages b/doc/website/release-notes/.pages new file mode 100644 index 0000000000..f5399ad187 --- /dev/null +++ b/doc/website/release-notes/.pages @@ -0,0 +1,6 @@ +nav: + - iceoryx-unreleased.md + - iceoryx-v2-0-0.md + - iceoryx-v1-0-2.md + - iceoryx-v1-0-1.md + - iceoryx-v1-0-0.md diff --git a/doc/website/release-notes/change-log.md b/doc/website/release-notes/change-log.md deleted file mode 100644 index 28c59db607..0000000000 --- a/doc/website/release-notes/change-log.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Changelog ---- - -{! ../iceoryx/CHANGELOG.md !} diff --git a/doc/website/release-notes/iceoryx-unreleased.md b/doc/website/release-notes/iceoryx-unreleased.md new file mode 100644 index 0000000000..bdc456c1ac --- /dev/null +++ b/doc/website/release-notes/iceoryx-unreleased.md @@ -0,0 +1,27 @@ +# iceoryx vx.x.x + +## [vx.x.x](https://github.com/eclipse-iceoryx/iceoryx/tree/vx.x.x) (xxxx-xx-xx) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/vx.x.x...vx.x.x) + +**Features:** + +**Bugfixes:** + +- Foo Bar [\#000](https://github.com/eclipse-iceoryx/iceoryx/issues/000) + +**Refactoring:** + +**New API features:** + +**API Breaking Changes:** + +1. Some API change. + + ```cpp + // before + #include "old/include.hpp" + + // after + #include "new/include.hpp" + ``` \ No newline at end of file diff --git a/doc/website/release-notes/iceoryx-v1-0-0.md b/doc/website/release-notes/iceoryx-v1-0-0.md new file mode 100644 index 0000000000..f82f5d2bd9 --- /dev/null +++ b/doc/website/release-notes/iceoryx-v1-0-0.md @@ -0,0 +1,297 @@ +# iceoryx v1.0.0 + +## [v1.0.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v1.0.0) (2021-04-15) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.90.7...v1.0.0) + +Description: +This is the first major release for Eclipse iceoryx. That means it is the first release with long-term support and the adopters of iceoryx can rely on a stable API. The release called Almond allows for true zero-copy inter-process-communication on Linux, QNX and MacOS and provides C and modern C++ user APIs. This release is supported until 2022-04-01. + +Compared to the feature content of the initial contribution, the main new features are: + +**Features:** + +- Introduction of a C API +- Refactoring of C++ API +- MacOS support +- Support for n:m communication +- Bridge to Eclipse Cyclone DDS + +See [Eclipse iceoryx 1.0.0 (Almond)](https://projects.eclipse.org/projects/technology.iceoryx/releases/1.0.0-almond) for more information. + +- Reserved chunkinfo user payload header [\#14](https://github.com/eclipse-iceoryx/iceoryx/issues/14) +- New chunk available callback for the new C++ and C APIs [\#350](https://github.com/eclipse-iceoryx/iceoryx/issues/350) +- Add generic QNX toolchain files [\#609](https://github.com/eclipse-iceoryx/iceoryx/issues/609) +- Introduction of Runnables/Nodes for the new APIs [\#349](https://github.com/eclipse-iceoryx/iceoryx/issues/349) + +**Refactoring:** + +- Added tests to iceoryx_posh and iceoryx_utils [\#496](https://github.com/eclipse-iceoryx/iceoryx/issues/496), + [\#484](https://github.com/eclipse-iceoryx/iceoryx/issues/484), [\#454](https://github.com/eclipse-iceoryx/iceoryx/issues/454) +, [\#240](https://github.com/eclipse-iceoryx/iceoryx/issues/240) +- Add [[nodiscard]] keyword to cxx::expected class [\#624](https://github.com/eclipse-iceoryx/iceoryx/issues/624) +- Refactor Relocatable Pointer [\#605](https://github.com/eclipse-iceoryx/iceoryx/issues/605) +- Integration test with RouDi and new API elements [\#378](https://github.com/eclipse-iceoryx/iceoryx/issues/378) +- Refactoring copyright headers [\#483](https://github.com/eclipse-iceoryx/iceoryx/issues/483) +- Integration of Unix Domain Sockets [\#381](https://github.com/eclipse-iceoryx/iceoryx/issues/381) +- Add how to install after build to guide [\#533](https://github.com/eclipse-iceoryx/iceoryx/issues/533) +- Add maven and openjdk-14-jdk-headless as build prerequisites [\#525](https://github.com/eclipse-iceoryx/iceoryx/issues/525) +- Refine modern pub/sub API [\#408](https://github.com/eclipse-iceoryx/iceoryx/issues/408) +- Replace introspection threads with PeriodicTask [\#489](https://github.com/eclipse-iceoryx/iceoryx/issues/489) +- Enable UndefinedBehaviorSanitizer [\#489](https://github.com/eclipse-iceoryx/iceoryx/issues/459) +- Move iceoryx from eclipse to iceoryx-eclipse organization [\#467](https://github.com/eclipse-iceoryx/iceoryx/issues/467) + +**Fixed bugs:** + +- TOML parser exceptions will crash RouDi [\#622](https://github.com/eclipse-iceoryx/iceoryx/issues/622) +- fail build on ubuntu 16.04 with g++ 5.4.0 [\#495](https://github.com/eclipse-iceoryx/iceoryx/issues/495) +- Wrong handling of applications that are started multiple times in parallel [\#404](https://github.com/eclipse-iceoryx/iceoryx/issues/404) +- Expected and_then/or_else calling callable without checking [\#599](https://github.com/eclipse-iceoryx/iceoryx/issues/599) +- std::void_t used in code [\#591](https://github.com/eclipse-iceoryx/iceoryx/issues/591) +- Fix resource leak caused by move ctor/assignment [\#542](https://github.com/eclipse-iceoryx/iceoryx/issues/542) +- nanoseconds in units::Duration are truncated [\#190](https://github.com/eclipse-iceoryx/iceoryx/issues/190) +- popo/user_trigger.hpp:33:5: error: exception specification of explicitly defaulted default constructor does not match the calculated one [\#494](https://github.com/eclipse-iceoryx/iceoryx/issues/494) +- iceoryx libs should be build as static libs [\#509](https://github.com/eclipse-iceoryx/iceoryx/issues/509) +- Build with gcov is broken [\#497](https://github.com/eclipse-iceoryx/iceoryx/issues/497) + +## [v0.99.7](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.7) (2021-04-09) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.6...v0.99.7) + +- cleanup testing libraries; if used outside of iceoryx, include paths and link targets must be adjusted +- build iceoryx_utils, iceoryx_posh and iceoryx_binding_c as shared libraries in colcon + +## [v0.99.6](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.6) (2021-04-08) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.5...v0.99.6) + +- Add git to dependency list in iceoryx_posh for RHEL + +## [v0.99.5](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.5) (2021-04-08) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.4...v0.99.5) + +- Add libatomic to dependency list in iceoryx_utils for RHEL + +## [v0.99.4](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.4) (2021-04-06) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.3...v0.99.4) + +- Increase version number to trigger build again on ROS buildfarm + +## [v0.99.3](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.3) (2021-04-06) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.2...v0.99.3) + +- Increase version number to trigger build again on ROS buildfarm + +## [v0.99.2](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.2) (2021-04-03) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.99.0...v0.99.2) + +- Prepare package release for iceoryx 1.0 [\#670](https://github.com/eclipse-iceoryx/iceoryx/issues/670) + +## [v0.99.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.99.0) (2021-04-01) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.90.0...v0.99.0) + +Intermediate Milestone before the final 1.0.0 release, see v.1.0.0 above for the changes. + +## [v0.90.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.90.0) (2020-12-22) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.17.0...v0.90.0) + +Pre-Release for new and stable APIs and n:m pub/sub communication + +This is a pre-release for our first long-term-support release that is coming soon (will be iceoryx_1.0.0). + +We had a major refactoring of the iceoryx communication infrastructure which allows to do n:m communication now and provides a flexible history functionality for late joining subscribers. + +This new infrastructure is also made to support request/response communication as a next step [(\#27)](https://github.com/eclipse-iceoryx/iceoryx/issues/27). The C++ API has been completely reworked and a new C API has been added. + +We will give the API draft some weeks for finalizing the features and to incorporate feedback from the community. There's an issue in which we collect and discuss the final modifications [(\#408)](https://github.com/eclipse-iceoryx/iceoryx/issues/408). + +**Features:** + +- Complete refactoring of publishers and subscribers from shared memory data structures to user APIs, [\#252](https://github.com/eclipse-iceoryx/iceoryx/issues/252) +- First versions of new APIs for C++ and C [\#252](https://github.com/eclipse-iceoryx/iceoryx/issues/252) +- n:m publish/subscribe communication now possible [\#25](https://github.com/eclipse-iceoryx/iceoryx/issues/25) +- First version of a DDS gateway. Cyclone DDS already integrated, FastDDS on it's way [\#64](https://github.com/eclipse-iceoryx/iceoryx/issues/64), [\#65](https://github.com/eclipse-iceoryx/iceoryx/issues/65) +- New github actions for Mac OS and colcon build [\#175](https://github.com/eclipse-iceoryx/iceoryx/issues/175), [\#276](https://github.com/eclipse-iceoryx/iceoryx/issues/276), [\#328](https://github.com/eclipse-iceoryx/iceoryx/issues/328) +- Adjustable capacity for the lockfree queue [\#216](https://github.com/eclipse-iceoryx/iceoryx/issues/216) +- Check the files have a copyright header [\#346](https://github.com/eclipse-iceoryx/iceoryx/issues/346) + +**Refactoring:** + +- Refactoring of waitset [\#341](https://github.com/eclipse-iceoryx/iceoryx/issues/341) +- create multi publisher example [\#394](https://github.com/eclipse-iceoryx/iceoryx/issues/394) +- Full IceOryx Public API Cheat Sheet [\#283](https://github.com/eclipse-iceoryx/iceoryx/issues/283) +- Rework build and test steps in iceoryx [\#433](https://github.com/eclipse-iceoryx/iceoryx/issues/433) +- Extend iceperf example to C API [\#453](https://github.com/eclipse-iceoryx/iceoryx/issues/453) +- Remove default parameter from PoshRuntime::getInstance() [\#382](https://github.com/eclipse-iceoryx/iceoryx/issues/382) +- Enable Sanitizer in Debug Build and Unit Tests [\#141](https://github.com/eclipse-iceoryx/iceoryx/issues/141) +- Minor RouDi cleanups [\#91](https://github.com/eclipse-iceoryx/iceoryx/issues/91) +- C++14 [\#220](https://github.com/eclipse-iceoryx/iceoryx/issues/220) +- Replace occurence of std::list by cxx::list [\#221](https://github.com/eclipse-iceoryx/iceoryx/issues/221) + +**Fixed bugs:** + +- SegFault in iox-roudi on startup [\#447](https://github.com/eclipse-iceoryx/iceoryx/issues/447) +- Mocks can cause segfaults/undefined behavior [\#427](https://github.com/eclipse-iceoryx/iceoryx/issues/427) +- Chunks are lost forever when having an overflow in a variant queue of type FIFO [\#456](https://github.com/eclipse-iceoryx/iceoryx/issues/456) +- RouDi does not free shared memory properly on x86 Linux [\#324](https://github.com/eclipse-iceoryx/iceoryx/issues/324) +- WaitSet behavior wrong in `waitAndReturnFulfilledConditions` [\#388](https://github.com/eclipse-iceoryx/iceoryx/issues/388) +- Incorrect usage of strncpy [\#374](https://github.com/eclipse-iceoryx/iceoryx/issues/374) +- Global Instantiation of Publisher/Subscriber created core dump [\#327](https://github.com/eclipse-iceoryx/iceoryx/issues/327) + +## [v0.17.0](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0) (2020-08-27) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC6...v0.17.0)\ +MacOS support and preparations for new API + +**Packages:** + +- iceoryx_posh (Quality level 4) +- iceoryx_utils (Quality level 4) +- iceoryx_examples (Quality level 5) + +**Features:** + +- MacOS support [\#32](https://github.com/eclipse-iceoryx/iceoryx/issues/32) +- Major RouDi refactorings [\#70](https://github.com/eclipse-iceoryx/iceoryx/issues/70), [\#59](https://github.com/eclipse-iceoryx/iceoryx/issues/59), [\#78](https://github.com/eclipse-iceoryx/iceoryx/issues/78) +- Preparations for new iceoryx API [\#25](https://github.com/eclipse-iceoryx/iceoryx/issues/25) +- iceoryx to cyclonedds gateway [\#64](https://github.com/eclipse-iceoryx/iceoryx/issues/64) +- Introduce cxx::function_ref [\#86](https://github.com/eclipse-iceoryx/iceoryx/issues/86) + +**Fixed bugs:** + +- POSIX timer improvements [\#120](https://github.com/eclipse-iceoryx/iceoryx/issues/120), [\#167](https://github.com/eclipse-iceoryx/iceoryx/issues/167) +- Memory Synchronisation Issue in FiFo [\#119](https://github.com/eclipse-iceoryx/iceoryx/issues/119) +- Roudi MessageQueue thread startup [\#171](https://github.com/eclipse-iceoryx/iceoryx/issues/171) +- Destructor fix MessageQueue and UnixDomainSocket [\#150](https://github.com/eclipse-iceoryx/iceoryx/issues/150) +- RouDi ressource clean-up [\#113](https://github.com/eclipse-iceoryx/iceoryx/issues/113) + +## [v0.17.0_RC6](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC6) (2020-07-29) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC5...ICEORYX_0.17.0_RC6)\ +Release Candidate for 0.17.0 release + +**Fixed bugs:** + +- Sporadic timing test failure [\#120](https://github.com/eclipse-iceoryx/iceoryx/issues/120) +- Improvement: Merge Mempool Introspection data into one sample [\#210](https://github.com/eclipse-iceoryx/iceoryx/issues/210) + +## [v0.17.0_RC5](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC5) (2020-07-29) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC4...ICEORYX_0.17.0_RC5)\ +Release Candidate for 0.17.0 release + +**Fixed bugs:** + +- Callable can outrun periodicity of POSIX timer [\#161](https://github.com/eclipse-iceoryx/iceoryx/issues/161) + +## [v0.17.0_RC4](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC4) (2020-07-29) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC3...ICEORYX_0.17.0_RC4)\ +Release Candidate for 0.17.0 release + +**Fixed bugs:** + +- The destroy method of MessageQueue and UnixDomainSocket does not fully invalidate the object [\#150](https://github.com/eclipse-iceoryx/iceoryx/issues/150) +- Roudi message queue thread startup [\#171](https://github.com/eclipse-iceoryx/iceoryx/issues/171) + +## [v0.17.0_RC3](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC3) (2020-07-27) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC2...ICEORYX_0.17.0_RC3)\ +Release Candidate for 0.17.0 release + +**Features:** + +- cmake options for deployment parameter [\#142](https://github.com/eclipse-iceoryx/iceoryx/issues/142) + +## [v0.17.0_RC2](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC2) (2020-07-27) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/ICEORYX_0.17.0_RC1...ICEORYX_0.17.0_RC2)\ +Release Candidate for 0.17.0 release + +**Refactoring:** + +- Refactoring of logging [\#88](https://github.com/eclipse-iceoryx/iceoryx/issues/88) +- Remove const_cast wherever possible [\#76](https://github.com/eclipse-iceoryx/iceoryx/issues/76) +- Remove asynchronous service discovery feature [\#90](https://github.com/eclipse-iceoryx/iceoryx/issues/90) +- qacpp-4.5.0-2427 [\#93](https://github.com/eclipse-iceoryx/iceoryx/issues/93) +- Usage of github actions to build pull requests [\#89](https://github.com/eclipse-iceoryx/iceoryx/issues/89) +- Reduce default memory consumption with config and mempools [\#78](https://github.com/eclipse-iceoryx/iceoryx/issues/78) + +**Fixed bugs:** + +- Sporadic timing test failure [\#120](https://github.com/eclipse-iceoryx/iceoryx/issues/120) +- Make SOFI real size equal to the specified one [\#105](https://github.com/eclipse-iceoryx/iceoryx/issues/105) +- increase padding in introspection [\#117](https://github.com/eclipse-iceoryx/iceoryx/issues/117) +- Increase the process waiting for RouDi timeout to 60 seconds [\#110](https://github.com/eclipse-iceoryx/iceoryx/issues/110) + +## [v0.17.0_RC1](https://github.com/eclipse-iceoryx/iceoryx/tree/ICEORYX_0.17.0_RC1) (2020-03-24) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.16.1...ICEORYX_0.17.0_RC1)\ +Release Candidate for 0.17.0 release + +**Features:** + +- Memory abstraction, modularization of RouDi and fixes [\#59](https://github.com/eclipse-iceoryx/iceoryx/issues/59) +- Memory abstraction and RouDi modularization step 2, fixed string improvements and other fixes [\#70](https://github.com/eclipse-iceoryx/iceoryx/issues/70) + +**Fixed bugs:** + +- Can't compile due to missing CPPTOML [\#67](https://github.com/eclipse-iceoryx/iceoryx/issues/67) + +## [v0.16.1](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.16.1) (2020-03-02) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v0.16.0...v0.16.1)\ +Support for rmw_iceoryx with ROS2 python API + +**Packages:** + +- POSH (POSIX Shared Memory) +- Utils +- Iceoryx examples + +**Features:** + +- Capability to cleanup shared memory resources during process lifetime [\#51](https://github.com/eclipse-iceoryx/iceoryx/issues/51) + +**Fixed bugs:** + +- compile error with clang [\#43](https://github.com/eclipse-iceoryx/iceoryx/issues/43) +- CMqInterface unit tests are failing caused by unneeded move operations [\#56](https://github.com/eclipse-iceoryx/iceoryx/issues/56) + +**Known limitations:** + +- ```RouDi --version``` shows 0.16.0.1 instead of 0.16.1 + +## [v0.16.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v0.16.0) (2019-12-16) + +Introspection, performance test and flexible mapping of shared memory + +**Packages:** + +- POSH (POSIX Shared Memory) +- Utils +- Iceoryx examples + +**Features:** + +- Introspection client for live debugging of iceoryx [\#21](https://github.com/eclipse-iceoryx/iceoryx/issues/21) +- Flexible mapping of shared memory in virtual address spaces [\#19](https://github.com/eclipse-iceoryx/iceoryx/issues/19) +- Performance test to measure inter-process latency [\#17](https://github.com/eclipse-iceoryx/iceoryx/issues/17) +- Docker build [\#15](https://github.com/eclipse-iceoryx/iceoryx/issues/15) (Thanks to @Mr-Slippery ) + +**Fixed bugs:** + +- Payload size not updated by allocateChunk [\#10](https://github.com/eclipse-iceoryx/iceoryx/issues/10) +- Failure in runnable creation [\#23](https://github.com/eclipse-iceoryx/iceoryx/issues/23) + +Command for generating git log for merge commits between two tags + +```bash +**git log --merges --first-parent master --pretty=format:"%h %<(10,trunc)%aN %C(white)%<(15)%aD%Creset %C(red bold)%<(15)%D%Creset %s" ... > diff_merge_commit.log** +``` diff --git a/doc/website/release-notes/iceoryx-v1-0-1.md b/doc/website/release-notes/iceoryx-v1-0-1.md new file mode 100644 index 0000000000..58df326b13 --- /dev/null +++ b/doc/website/release-notes/iceoryx-v1-0-1.md @@ -0,0 +1,19 @@ +# iceoryx v.1.0.1 + +## [v1.0.1](https://github.com/eclipse-iceoryx/iceoryx/tree/v1.0.0) (2021-06-15) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v1.0.0...v1.0.1) + +**Description:** +This is the first bugfix release for Eclipse iceoryx 1.0.0. We made minor changes in the documentation and added several patches. + +Compared to the feature content of the release 1.0.0, the following bug tickets where resolved: + +**Bugfixes:** + +- Github CI Integrationtests failing [\#825](https://github.com/eclipse-iceoryx/iceoryx/issues/825) +- Segmentation Fault in FileLock [\#808](https://github.com/eclipse-iceoryx/iceoryx/issues/808) +- cpptoml cannot be build due to failing examples [\#791](https://github.com/eclipse-iceoryx/iceoryx/issues/791) +- Missing template parameter for user-header in typed C++ API [\#768](https://github.com/eclipse-iceoryx/iceoryx/issues/768) +- Fix issues with generating website documentation [\#743](https://github.com/eclipse-iceoryx/iceoryx/issues/743) +- Starting an application after RouDi was terminated floods the cmd line with error messages [\#654](https://github.com/eclipse-iceoryx/iceoryx/issues/654) diff --git a/doc/website/release-notes/iceoryx-v1-0-2.md b/doc/website/release-notes/iceoryx-v1-0-2.md new file mode 100644 index 0000000000..8122819585 --- /dev/null +++ b/doc/website/release-notes/iceoryx-v1-0-2.md @@ -0,0 +1,9 @@ +# iceoryx v.1.0.2 + +## [v1.0.2](https://github.com/eclipse-iceoryx/iceoryx/tree/v1.0.2) (2022-02-18) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v1.0.1...v1.0.2) + +**Bugfixes:** + +- Backporting Bugfixes for GCC 11 [\#1092](https://github.com/eclipse-iceoryx/iceoryx/issues/1092) diff --git a/doc/website/release-notes/iceoryx-v2-0-0.md b/doc/website/release-notes/iceoryx-v2-0-0.md new file mode 100644 index 0000000000..f24a73a04a --- /dev/null +++ b/doc/website/release-notes/iceoryx-v2-0-0.md @@ -0,0 +1,346 @@ +# iceoryx v2.0.0 + +## [v2.0.0](https://github.com/eclipse-iceoryx/iceoryx/tree/v2.0.0) (2022-02-28) + +[Full Changelog](https://github.com/eclipse-iceoryx/iceoryx/compare/v1.0.2...v2.0.0) + +**Features:** + +- Partial enable iceoryx building with msvc2015+clang [\#965](https://github.com/eclipse-iceoryx/iceoryx/issues/965) +- C binding for posh configuration [\#930](https://github.com/eclipse-iceoryx/iceoryx/issues/930) +- Enhance MacOS performance with timed{send,receive} functionality in unix domain socket[\#903](https://github.com/eclipse-iceoryx/iceoryx/issues/903) +- Multi-Publisher support for DDS gateway and generic gateway class [\#900](https://github.com/eclipse-iceoryx/iceoryx/issues/900) +- Replace `iox-gw-iceoryx2dds` and `iox-gw-dds2iceoryx` gateways with `iox-dds-gateway` [\#900](https://github.com/eclipse-iceoryx/iceoryx/issues/900) +- Enhance `posixCall` [\#805](https://github.com/eclipse-iceoryx/iceoryx/issues/805) +- New chunk available callback for the new C++ [\#391](https://github.com/eclipse-iceoryx/iceoryx/issues/391) +- Git Hooks on iceoryx [\#486](https://github.com/eclipse-iceoryx/iceoryx/issues/486) +- static memory alternative for std::function [\#391](https://github.com/eclipse-iceoryx/iceoryx/issues/391) +- Adding support for Helix QAC 2021.1 [\#755](https://github.com/eclipse-iceoryx/iceoryx/issues/755) thanks to @toniglandy1 +- Axivion analysis on CI [\#409](https://github.com/eclipse-iceoryx/iceoryx/issues/409) +- Cpptoml can be provided by an external source [\#951](https://github.com/eclipse-iceoryx/iceoryx/issues/) +- Extend `cxx::optional` constructor for in place construction so that copy/move for values inside the optional even could be deleted [\#967](https://github.com/eclipse-iceoryx/iceoryx/issues/967) +- Add templated `from`/`into` free functions to formalize conversions from enums and other types [#992](https://github.com/eclipse-iceoryx/iceoryx/issues/992) +- `UniqueId` class for unique IDs within a process [#1010](https://github.com/eclipse-iceoryx/iceoryx/issues/1010) +- Add `requirePublisherHistorySupport` option at subscriber side (if set to true requires historyRequest <= historyCapacity to be eligible for connection) [#1029](https://github.com/eclipse-iceoryx/iceoryx/issues/1029) +- Add `/tools/scripts/ice_env.sh` shell script to provide simple access to docker containers for CI debugging [#1049](https://github.com/eclipse-iceoryx/iceoryx/issues/1049) +- Introduce `cxx::FunctionalInterface` to enrich nullable classes with `and_then`, `or_else`, `value_or`, `expect` [\#996](https://github.com/eclipse-iceoryx/iceoryx/issues/996) +- Add C++17 `std::perms` as `cxx::perms` to `iceoryx_hoofs/cxx/filesystem.hpp`. [#1059](https://github.com/eclipse-iceoryx/iceoryx/issues/1059) +- Support FreeBSD as a representative for the UNIX platforms [#1054](https://github.com/eclipse-iceoryx/iceoryx/issues/1054) +- Add event parameter to `findService` method [#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) +- Implement stream operator for `ChunkReceiveResult` and `AllocationError` to be able to use it with ostream and LogStream [\#1062](https://github.com/eclipse-iceoryx/iceoryx/issues/1062) +- Replace IPC-channel-based `findService` with pub/sub-based on [#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) + +**Bugfixes:** + +- Fix support for libc++ on clang [\#905](https://github.com/eclipse-iceoryx/iceoryx/issues/905) +- Fix warnings for gcc-11.1 [\#838](https://github.com/eclipse-iceoryx/iceoryx/issues/838) +- Incremental builds with the build script are broken [\#821](https://github.com/eclipse-iceoryx/iceoryx/issues/821) +- Compile failed because of missing `` for GCC 11 [\#811](https://github.com/eclipse-iceoryx/iceoryx/issues/811) thanks to @homalozoa +- Unable to build cyclone dds `idlpp-cxx` [\#736](https://github.com/eclipse-iceoryx/iceoryx/issues/736) +- Fix format string issues with introspection client [\#960](https://github.com/eclipse-iceoryx/iceoryx/issues/960) thanks to @roehling +- Add support for Multi-Arch install destinations [\#961](https://github.com/eclipse-iceoryx/iceoryx/issues/961) thanks to @roehling +- Fix a few misspellings in log messages [\#962](https://github.com/eclipse-iceoryx/iceoryx/issues/962) thanks to @roehling +- Fix typos in goals/non-goals document [\#968](https://github.com/eclipse-iceoryx/iceoryx/issues/968) +- Catch deserialization errors for enums in publisher and subscriber options [\#989](https://github.com/eclipse-iceoryx/iceoryx/issues/989) +- Fix linker error on QNX [\#1013](https://github.com/eclipse-iceoryx/iceoryx/issues/1013) +- When posix mutex fails a correct error message is reported on the console [\#999](https://github.com/eclipse-iceoryx/iceoryx/issues/999) +- Only use `std::result_of` for C++14 to be able to use iceoryx in C++20 projects [\#1076](https://github.com/eclipse-iceoryx/iceoryx/issues/1076) +- Set stack size for windows in `singleprocess` example and posh tests [\#1082](https://github.com/eclipse-iceoryx/iceoryx/issues/1082) + +**Refactoring:** + +- Handle nullptr callbacks in waitset and listener [\#932](https://github.com/eclipse-iceoryx/iceoryx/issues/932) +- Add clang-tidy rules for iceoryx_hoofs [\#889](https://github.com/eclipse-iceoryx/iceoryx/issues/889) +- Move all tests into an anonymous namespace [\#563](https://github.com/eclipse-iceoryx/iceoryx/issues/563) +- Refactor `smart_c` to use contract by design and expected [\#418](https://github.com/eclipse-iceoryx/iceoryx/issues/418) +- `PoshRuntime` Mock [\#449](https://github.com/eclipse-iceoryx/iceoryx/issues/449) +- Clean-up Doxygen for all methods in headers [\#583](https://github.com/eclipse-iceoryx/iceoryx/issues/583) +- Rename utils to hoofs [\#790](https://github.com/eclipse-iceoryx/iceoryx/pull/790) +- plantuml in design documentation [\#787](https://github.com/eclipse-iceoryx/iceoryx/pull/787) +- Refine quality levels [\#425](https://github.com/eclipse-iceoryx/iceoryx/issues/425) +- Clean-up std::terminate usage [\#261](https://github.com/eclipse-iceoryx/iceoryx/issues/261) +- Add Quality Declaration Document [\#910](https://github.com/eclipse-iceoryx/iceoryx/issues/910) +- Make `cxx::string::capacity` a `static` method [\#979](https://github.com/eclipse-iceoryx/iceoryx/issues/979) +- Restructure iceoryx tools [\#471](https://github.com/eclipse-iceoryx/iceoryx/issues/471) +- Use cxx::expected for MemoryManager::getChunk [\#954](https://github.com/eclipse-iceoryx/iceoryx/pull/991) +- Upgrade GTest/GMock to v1.10 [\#841](https://github.com/eclipse-iceoryx/iceoryx/issues/841) +- Remove the requirement for INVALID_STATE for the cxx::expected [\#987](https://github.com/eclipse-iceoryx/iceoryx/issues/987) +- Add unique test identifers [\#988](https://github.com/eclipse-iceoryx/iceoryx/issues/988) +- Remove `ApplicationPort` and `ApplicationPortData` classes [\#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) +- Remove creation pattern from `MemoryMap` and replace it with `MemoryMapBuilder` [\#1036](https://github.com/eclipse-iceoryx/iceoryx/issues/1036) +- Fix error handling of `TypedUniqueId` and refactor it to `UniquePortId` [\#861](https://github.com/eclipse-iceoryx/iceoryx/issues/861) +- Updating Codecov API and enforce CMake version 3.16 for building iceoryx [\#774](https://github.com/eclipse-iceoryx/iceoryx/issues/774) and [\#1031](https://github.com/eclipse-iceoryx/iceoryx/issues/1031) +- Remove `InvalidIdString` and `isValid()` from `ServiceDescription`, replace Wildcard string with `iox::cxx::nullopt` [\#415](https://github.com/eclipse-iceoryx/iceoryx/issues/415) +- Remove creation pattern from `SharedMemory` and replace it with `SharedMemoryBuilder` [\#1036](https://github.com/eclipse-iceoryx/iceoryx/issues/1036) +- Remove the leading slash requirement from the name of a shared memory in `SharedMemory` and `SharedMemoryObject` [\#439](https://github.com/eclipse-iceoryx/iceoryx/issues/439) + +**New API features:** + +1. Introduce `iceoryx_hoofs/cxx/filesystem.hpp` which implements `std::perms` as `cxx::perms`. + + ```cpp + #include "iceoryx_hoofs/cxx/filesystem.hpp" + + // ... + cxx::perms filePermissions; + filePermissions = cxx::perms::owner_read | cxx::perms::group_write; + std::cout << filePermissions << std::endl; + ``` + +**API Breaking Changes:** + +1. The CMake files in iceoryx expect to have CMake version 3.16 or greater installed, otherwise the build fails. + (Hint: Ubuntu 18 users can install `cmake-mozilla` from the universe repository provided by Canonical) + +1. Dependency for building the `iceoryx_dds` gateway changed from `openjdk` (Java) to `bison` + +1. Change include from `iceoryx_hoofs/cxx/helplets.hpp` to `iceoryx_hoofs/cxx/requires.hpp` + when using `cxx::Expects` or `cxx::Ensures` + + ```cpp + // before + #include "iceoryx_hoofs/cxx/helplets.hpp" + + iox::cxx::Expects(someCondition); + iox::cxx::Ensures(anotherCondition); + + // after + #include "iceoryx_hoofs/cxx/requires.hpp" + + iox::cxx::Expects(someCondition); + iox::cxx::Ensures(anotherCondition); + ``` + +1. Replace Creation pattern from `MemoryMap` with `MemoryMapBuilder`. + + ```cpp + // before + auto memoryMapResult = posix::MemoryMap::create(baseAddress, length, fileDescriptor, accessMode, flags, offset); + + // after + auto memoryMapResult = posix::MemoryMapBuilder().baseAddressHint(baseAddress) + .length(length).fileDescriptor(fileDescriptor) + .accessMode(accessMode).flags(flags) + .offset(0).create(); + ``` + +1. Rename utils to hoofs: + + In CMake you need now to find and link the package `iceoryx_hoofs` instead of `iceoryx_utils` + + ```cmake + # before + find_package(iceoryx_utils REQUIRED) + target_link_libraries(${target} + iceoryx_utils::iceoryx_utils) + + # after + find_package(iceoryx_hoofs REQUIRED) + target_link_libraries(${target} + iceoryx_hoofs::iceoryx_hoofs) + ``` + + The include paths for `iceoryx_utils` are now `iceoryx_hoofs` + + ```cpp + // before + #include "iceoryx_utils/cxx/string.hpp" + + // after + #include "iceoryx_hoofs/cxx/string.hpp" + ``` + +1. Refactoring SmartC: + + - Renaming SmartC wrapper to posixCall. + - Removed `getErrorString()` from posixCall, please use `getHumanReadableErrnum()` instead. + - Enhanced posixCall to handle a common case were multiple errnos are ignored just to suppress error logging + + ```cpp + // before + #include "iceoryx_utils/cxx/smart_c.hpp" + + auto unlinkCallPublisher = iox::cxx::makeSmartC( + unlink, iox::cxx::ReturnMode::PRE_DEFINED_ERROR_CODE, {ERROR_CODE}, {ENOENT}, sockAddrPublisher.sun_path); + + if (unlinkCallPublisher.hasErrors()) + { + std::cout << "unlink error" << std::endl; + exit(1); + } + + // after + #include "iceoryx_utils/posix_wrapper/posix_call.hpp" + + iox::posix::posixCall(unlink)(sockAddrPublisher.sun_path) + .failureReturnValue(ERROR_CODE) + .ignoreErrnos(ENOENT, EBUSY) // can be a comma-separated list of errnos + .evaluate() + .or_else([](auto& r) { + std::cout << "unlink error " << r.getHumanReadableErrnum() << std::endl; + exit(1); + }); + ``` + +1. Refactoring of `ServiceDescription` + + - A `ServiceDescription` is now only string-based and no more wildcards are allowed. + - A well-defined `ServiceDescription` consists of three non-empty strings. + + ```cpp + // before + ServiceDescription myServiceDescription1(1U, 2U, 3U); + ServiceDescription myServiceDescription3("First", "Second"); + ServiceDescription myServiceDescription3(iox::capro::AnyServiceString, iox::capro::AnyInstanceString, iox::capro::AnyEventString); + + // after + ServiceDescription myServiceDescription1("Foo", "Bar", "Baz"); + ServiceDescription myServiceDescription2("First", "Second", "DontCare"); + ServiceDescription myServiceDescription3("Foo", "Bar", "Baz"); + ``` + +1. Instead of using a constructor a `ServiceDescription` is now deserialized via a + static method with error handling: + + ```cpp + // before + iox::cxx::Serialization serializedObj; + iox::capro::ServiceDescription service(serializedObj); + + // after + iox::cxx::Serialization serialisedObj; + capro::ServiceDescription::deserialize(serialisedObj) + .and_then([](auto& value){ + // Do something with the deserialized object + }) + .or_else([](auto& error){ + // Handle the error + }); + ``` + +1. The `InvalidIdString` was removed from `ServiceDescription` and the Wildcard string was replaced + with a `iox::cxx::nullopt`. With this, every string is allowed within the `ServiceDescription`. + The default `ServiceDescription` consists of empty strings. + +1. The service-related methods have been moved from `PoshRuntime` to `ServiceDiscovery`. The `offerService` + and `stopOfferService` methods have been removed and `findService` has now an additional event parameter: + + ```cpp + // before + #include "iceoryx_posh/runtime/posh_runtime.hpp" + + poshRuntime.offerService(myServiceDescription); + poshRuntime.stopOfferService(myServiceDescription); + poshRuntime.findService({"ServiceA", iox::capro::AnyInstanceString}); + + // after + #include "iceoryx_posh/runtime/service_discovery.hpp" + + serviceDiscovery.findService("ServiceA", Wildcard, Wildcard); + ``` + +1. The following classes have now an constructor marked as `explicit`: + + ```cpp + explicit DeadlineTimer(const iox::units::Duration timeToWait); + explicit GenericRAII(const std::function& cleanupFunction); + explicit mutex(const bool f_isRecursive); + explicit PosixUser(const uid_t f_id); + explicit PosixUser(const string_t& f_name); + ``` + +1. Renaming in `FileReader` class and logging of iceoryx_hoofs + + ```cpp + // before + iox::cxx::FileReader reader("filename"); + std::string str; + if(reader.IsOpen()) { + reader.ReadLine(str); + } + + static auto& logger = CreateLogger("", "", iox::log::LogManager::GetLogManager().DefaultLogLevel()); + + // after + iox::cxx::FileReader reader("filename"); + std::string str; + if(reader.isOpen()) { + reader.readLine(str); + } + + static auto& logger = createLogger("", "", iox::log::LogManager::GetLogManager().DefaultLogLevel()); + ``` + +1. The `iox::cxx::expected` has dropped the requirement for `INVALID_STATE`. With this, the + `ErrorTypeAdapter` which was necessary for non enum types was also removed. The specialization + of `ErrorTypeAdapter` for custom types must therefore also be removed in the user code. + +1. The queue port policy enums are adjusted to use them with `Client` and `Server`. + +1. The `QueueFullPolicy::BLOCK_PUBLISHER` is replaced with the more generic `QueueFullPolicy::BLOCK_PRODUCER`. + + ```cpp + // old + iox::popo::SubscriberOptions options; + options.queueFullPolicy = iox::popo::QueueFullPolicy::BLOCK_PUBLISHER; + + // new + iox::popo::SubscriberOptions options; + options.queueFullPolicy = iox::popo::QueueFullPolicy::BLOCK_PRODUCER; + ``` + + Similar, for `binding_c` it is `QueueFullPolicy_BLOCK_PRODUCER` instead of `QueueFullPolicy_BLOCK_PUBLISHER` + + ```c + // old + iox_sub_options_t options; + iox_sub_options_init(&options); + options.queueFullPolicy = QueueFullPolicy_BLOCK_PUBLISHER; + + // new + iox_sub_options_t options; + iox_sub_options_init(&options); + options.queueFullPolicy = QueueFullPolicy_BLOCK_PRODUCER; + ``` + +1. The `SubscriberTooSlowPolicy` is replaced with the more generic `ConsumerTooSlowPolicy` and + `SubscriberTooSlowPolicy::WAIT_FOR_SUBSCRIBER` became `ConsumerTooSlowPolicy::WAIT_FOR_CONSUMER`. + + ```cpp + // old + iox::popo::PublisherOptions options; + options.subscriberTooSlowPolicy = iox::popo::SubscriberTooSlowPolicy::WAIT_FOR_SUBSCRIBER; + + // new + iox::popo::PublisherOptions options; + options.subscriberTooSlowPolicy = iox::popo::ConsumerTooSlowPolicy::WAIT_FOR_CONSUMER; + ``` + + And with the `binding_c` + + ```c + // old + iox_pub_options_t options; + iox_pub_options_init(&options); + options.subscriberTooSlowPolicy = SubscriberTooSlowPolicy_WAIT_FOR_SUBSCRIBER; + + // new + iox_pub_options_t options; + iox_pub_options_init(&options); + options.subscriberTooSlowPolicy = ConsumerTooSlowPolicy_WAIT_FOR_CONSUMER; + ``` + +1. The `CaproMessageSubType` enum is renamed to `CaproServiceType` and the values are renamed from `NOSUBTYPE`, + `SERVICE`, `EVENT` and `FIELD` to `NONE`, `PUBLISHER` and `SERVER`. + This change only affects `InterfacePorts` which used this enum to communicate whether the `CaproMessage` + was from a `SERVICE`, `EVENT` or `FIELD`. This was quite ara::com specific and with the introduction of the `ServerPort` + changes were needed. The distinction between a `FIELD` and an `EVENT` can be made by checking + `CaproMessage::m_historyCapacity`. + + ```cpp + // old + caproMessage.m_subType = CaproMessageSubType::EVENT; + + // new + caproMessage.m_serviceType = CaproServiceType::PUBLISHER; + ``` From ce2272a02144e8b7b6feda96df5c476c03386642 Mon Sep 17 00:00:00 2001 From: Dietrich Kroenke Date: Thu, 17 Feb 2022 10:41:13 +0100 Subject: [PATCH 07/12] iox-#743 Adapt docs to new changelog location --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- CONTRIBUTING.md | 3 ++- QUALITY_DECLARATION.md | 2 +- iceoryx_meta/install_help_and_config.cmake | 6 +++++- tools/scripts/iceoryx_release.sh | 2 +- 5 files changed, 10 insertions(+), 5 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 8f94a4b040..c3f0131668 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -20,7 +20,7 @@ [eca]: http://www.eclipse.org/legal/ECA.php [contributing]: https://github.com/eclipse-iceoryx/iceoryx/blob/master/CONTRIBUTING.md#coding-style [testing]: https://github.com/eclipse-iceoryx/iceoryx/blob/master/doc/website/advanced/best-practice-for-testing.md -[changelog]: https://github.com/eclipse-iceoryx/iceoryx/blob/master/CHANGELOG.md +[changelog]: https://github.com/eclipse-iceoryx/iceoryx/blob/master/doc/website/release-notes/iceoryx-unreleased.md ## Notes for Reviewer diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 83eff907b5..516bb84d54 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -53,7 +53,8 @@ Please make sure you have: 3. All branches have the following naming format: `iox-#[issue]-branch-name` e.g. `iox-#123-super-duper-feature` 4. All commits have the following naming format: `iox-#[issue] commit message` e.g. `iox-#123 implemented super-duper feature` 5. All commits have been signed with `git commit -s` -6. The CHANGELOG.md is updated with the GitHub issue that is closed by the Pull-Request (add to unreleased section) +6. The `iceoryx-unreleased.md` in `doc/website/release-notes` is updated with the GitHub issue + that is closed by the Pull-Request 7. You open your pull request towards the base branch `master` 8. Link the pull request to the according GitHub issue and set the label accordingly diff --git a/QUALITY_DECLARATION.md b/QUALITY_DECLARATION.md index 14dc227920..99aef06fcc 100644 --- a/QUALITY_DECLARATION.md +++ b/QUALITY_DECLARATION.md @@ -23,7 +23,7 @@ On Git, the tags have a `v` prefix before the version numbers. A [release script `iceoryx` is at a stable version, i.e. `>= 1.0.0`. The latest valid release can be found on the [release page](https://github.com/eclipse-iceoryx/iceoryx/releases) of iceoryx. -The change history can be found in the [CHANGELOG](https://github.com/eclipse-iceoryx/iceoryx/blob/master/CHANGELOG.md). +The change history can be found in the [release notes section](https://iceoryx.io/latest/release-notes). ### Public API Declaration [1.iii] diff --git a/iceoryx_meta/install_help_and_config.cmake b/iceoryx_meta/install_help_and_config.cmake index 63946c038b..a435d6a1ef 100644 --- a/iceoryx_meta/install_help_and_config.cmake +++ b/iceoryx_meta/install_help_and_config.cmake @@ -22,7 +22,6 @@ install( ${CMAKE_CURRENT_SOURCE_DIR}/../LICENSE ${CMAKE_CURRENT_SOURCE_DIR}/../NOTICE.md ${CMAKE_CURRENT_SOURCE_DIR}/../README.md - ${CMAKE_CURRENT_SOURCE_DIR}/../CHANGELOG.md DESTINATION share/doc/iceoryx COMPONENT dev) @@ -30,3 +29,8 @@ install( DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/../doc/3rd_party_licenses DESTINATION share/doc/iceoryx COMPONENT dev) + + install( + DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/../doc/website/release-notes + DESTINATION share/doc/iceoryx + COMPONENT dev) diff --git a/tools/scripts/iceoryx_release.sh b/tools/scripts/iceoryx_release.sh index b4a7d36e66..4a7f1e8f92 100755 --- a/tools/scripts/iceoryx_release.sh +++ b/tools/scripts/iceoryx_release.sh @@ -22,7 +22,7 @@ set -e WORKSPACE=$(git rev-parse --show-toplevel) -echo "Did you updated the Changelog.md?" +echo "Did you updated the changelogs?" select yn in "Yes" "No"; do case $yn in Yes ) break;; From 5e02ca5ed5d7ff2c33d68cfceaf44082a2f8fc21 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Thu, 17 Feb 2022 15:03:12 +0100 Subject: [PATCH 08/12] iox-#743 Add autolink plugin for MkDocs and move images into one folder Signed-off-by: Simon Hoinkis --- doc/design/request_response_communication.md | 18 +- doc/design/service-discovery.md | 10 +- ...nd-error-values-are-returned-in-iceoryx.md | 125 +++++++ doc/website/design/chunk_header.md | 287 ++++++++++++++++ doc/website/design/error-handling.md | 318 ++++++++++++++++++ doc/website/design/listener.md | 317 +++++++++++++++++ doc/website/design/lockfree_queue.md | 135 ++++++++ doc/website/design/relocatale_pointer.md | 215 ++++++++++++ .../design/request_response_communication.md | 131 ++++++++ .../images}/client_and_server_options.svg | 0 .../client_and_server_service_discovery.svg | 0 .../images}/client_port.svg | 0 .../images}/client_state_machine.svg | 0 doc/website/images/favicon.ico | Bin 0 -> 15406 bytes ...ryx_components_diagram_nextgen_release.svg | 0 .../images}/overview-alternative-b.svg | 0 .../images}/overview-alternative-d.svg | 0 .../images}/overview_class.svg | 0 .../images}/puml_github_workflow.svg | 0 .../images}/request_response_header.svg | 0 .../sequence-diagram-alternative-b.svg | 0 .../sequence-diagram-alternative-d.svg | 0 .../images}/server_port.svg | 0 .../images}/typed_api.svg | 0 .../images}/untyped_api.svg | 0 mkdocs.yml | 6 +- tools/website/export-docu-to-website.sh | 3 +- 27 files changed, 1549 insertions(+), 16 deletions(-) create mode 100644 doc/website/concepts/how-optional-and-error-values-are-returned-in-iceoryx.md create mode 100644 doc/website/design/chunk_header.md create mode 100644 doc/website/design/error-handling.md create mode 100644 doc/website/design/listener.md create mode 100755 doc/website/design/lockfree_queue.md create mode 100644 doc/website/design/relocatale_pointer.md create mode 100644 doc/website/design/request_response_communication.md rename doc/{design/diagrams/request_response => website/images}/client_and_server_options.svg (100%) rename doc/{design/diagrams/request_response => website/images}/client_and_server_service_discovery.svg (100%) rename doc/{design/diagrams/request_response => website/images}/client_port.svg (100%) rename doc/{design/diagrams/request_response => website/images}/client_state_machine.svg (100%) create mode 100644 doc/website/images/favicon.ico rename doc/{design/diagrams => website/images}/iceoryx_components_diagram_nextgen_release.svg (100%) rename doc/{design/diagrams/service-discovery => website/images}/overview-alternative-b.svg (100%) rename doc/{design/diagrams/service-discovery => website/images}/overview-alternative-d.svg (100%) rename doc/{design/diagrams/request_response => website/images}/overview_class.svg (100%) rename doc/{design/diagrams => website/images}/puml_github_workflow.svg (100%) rename doc/{design/diagrams/request_response => website/images}/request_response_header.svg (100%) rename doc/{design/diagrams/service-discovery => website/images}/sequence-diagram-alternative-b.svg (100%) rename doc/{design/diagrams/service-discovery => website/images}/sequence-diagram-alternative-d.svg (100%) rename doc/{design/diagrams/request_response => website/images}/server_port.svg (100%) rename doc/{design/diagrams/request_response => website/images}/typed_api.svg (100%) rename doc/{design/diagrams/request_response => website/images}/untyped_api.svg (100%) diff --git a/doc/design/request_response_communication.md b/doc/design/request_response_communication.md index 64d6c61b55..e268a5da98 100644 --- a/doc/design/request_response_communication.md +++ b/doc/design/request_response_communication.md @@ -40,26 +40,26 @@ In order to support asynchronous requests, a sequence ID should be part of each This is an overview of the untyped `Client` and `Server` classes. -![simple class diagram](diagrams/request_response/overview_class.svg) +![simple class diagram](overview_class.svg) The `Client` and `Server` are reusing the `ChunkSender` and `ChunkReceiver` building blocks. The `Client` uses a `ChunkSender` to send requests and a `ChunkReceiver` to get the responses while the `Server` uses a `ChunkReceiver` to get the requests and a `ChunkSender` to send the responses. #### Typed API -![typed API](diagrams/request_response/typed_api.svg) +![typed API](typed_api.svg) Since the `Response` is tied to a specific `Request`, the `loan` method takes a `Request` to populate the `Response` with the correct settings. Alternatively, a `Request` could have a `createResponse` method which returns a `Response` with the correct settings. #### Untyped API -![untyped API](diagrams/request_response/untyped_api.svg) +![untyped API](untyped_api.svg) Similar to the the typed API, `loan` takes a pointer to a `RequestHeader` to populate the `ResponseHeader` with the correct settings. #### Client Port -![client port](diagrams/request_response/client_port.svg) +![client port](client_port.svg) The `ClientPortData` is located in the shared memory and contain only the data but no methods to access them. `ClientPortUser` is the class providing the methods for the user access and `ClientPortRouDi` provides the @@ -67,7 +67,7 @@ interface RouDi needs to connect the client to the server and to cleanup the por #### Server Port -![server port](diagrams/request_response/server_port.svg) +![server port](server_port.svg) Similar to the Client Port, the Server Port has `ServerPortData` which is located in the shared memory and contain only the data but no methods to access them. `ServerPortUser` is the class providing the methods for the user access and `ServerPortRouDi` provides the @@ -77,7 +77,7 @@ It must be ensured that only one server with a given `ServiceDescription` can ru #### Request/Response Header -![rpc header](diagrams/request_response/request_response_header.svg) +![rpc header](request_response_header.svg) Since request and response need to encode different meta-information, we also need different header for the messages. The common data is aggregated in `RpcBaseHeader` which contains a `cxx::UniqueId` to the `ClientChunkQueueData_t` and a sequence ID. @@ -101,7 +101,7 @@ The `RequestHeader` has also the option to specify a message as fire and forget, #### Client/Server Options -![client and server options](diagrams/request_response/client_and_server_options.svg) +![client and server options](client_and_server_options.svg) The client and server options can be used to control certain aspects of the clients and servers. Beside setting the capacity of the queues and defining whether a client should be connected and a server offering on creation, @@ -114,14 +114,14 @@ If the options don't match, the client will not be connected to the server, simi The client is guided by the following state machine. -![client state machine](diagrams/request_response/client_state_machine.svg) +![client state machine](client_state_machine.svg) Similar to the subscriber state machine, the client passes it's response queue with the `CONNECT` CaPro message to the server. The server will pass its request queue with the `ACK` CaPro message to the client. Following is a sequence diagram which shows all this cases -![client and server service discovery](diagrams/request_response/client_and_server_service_discovery.svg) +![client and server service discovery](client_and_server_service_discovery.svg) ### Code example diff --git a/doc/design/service-discovery.md b/doc/design/service-discovery.md index 64e67e4df1..905d27d7f3 100644 --- a/doc/design/service-discovery.md +++ b/doc/design/service-discovery.md @@ -128,9 +128,9 @@ Contra: * Built-in topic approach based on `InterfacePort`'s * Sniff and intercept `CaproMessage` -![overview diagram](diagrams/service-discovery/overview-alternative-b.svg) +![overview diagram](overview-alternative-b.svg) -![sequence diagram](diagrams/service-discovery/sequence-diagram-alternative-b.svg) +![sequence diagram](sequence-diagram-alternative-b.svg) Pro: @@ -172,9 +172,9 @@ Create a new publisher in RouDi which sends a `ServiceRegistryTopic`. This publi change in the service registry and to transmit the service discovery registry. The complete old service registry (saved locally) would be compared to the new service registry in a new class, extending the public user API. -![overview diagram](diagrams/service-discovery/overview-alternative-d.svg) +![overview diagram](overview-alternative-d.svg) -![sequence diagram](diagrams/service-discovery/sequence-diagram-alternative-d.svg) +![sequence diagram](sequence-diagram-alternative-d.svg) Pro: @@ -416,5 +416,5 @@ PoshRuntime::findService(const capro::ServiceDescription& serviceDescription) no * [x] How does the SOME/IP-SD service discovery API look like? * [x] What does a `ros topic list` do in rmw_iceoryx? * [ ] Filter for `ServiceDescription::EventString` needed by AUTOSAR? Not supported by `ServiceRegistry::find()` as of today -* [ ] Decision on mapping table between iceory and DDS (see [overview.md](../website/getting-started/overview.md)) +* [ ] Decision on mapping table between iceory and DDS (see [Overview](overview.md)) * [ ] Current mapping table between iceoryx and DDS does not work with service discovery diff --git a/doc/website/concepts/how-optional-and-error-values-are-returned-in-iceoryx.md b/doc/website/concepts/how-optional-and-error-values-are-returned-in-iceoryx.md new file mode 100644 index 0000000000..8396fb0c9b --- /dev/null +++ b/doc/website/concepts/how-optional-and-error-values-are-returned-in-iceoryx.md @@ -0,0 +1,125 @@ +# How optional and error values are returned in iceoryx + +Many parts of the iceoryx C++ API follow a functional programming approach and allow the user to specify functions +which handle the possible cases, e.g. what should happen when data is received. + +This is very flexible but requires using the monadic types `cxx::expected` and `cxx::optional`, which we +introduce in the following sections. + +## Optional + +The type `iox::cxx::optional` is used to indicate that there may or may not be a value of a specific type `T` +available. This is essentially the 'maybe [monad](https://en.wikipedia.org/wiki/Monad_(functional_programming))' in +functional programming. Assuming we have some optional (usually the result of some computation) + +```cpp +optional result = someComputation(); +``` + +we can check for its value using + +```cpp +if(result.has_value()) +{ + auto value = result.value(); + // do something with the value +} +else +{ + // handle the case that there is no value +} +``` + +A shorthand to get the value is + +```cpp +auto value = *result; +``` + +!!! attention + Accessing the value if there is no value terminates the application, so it must be checked beforehand. + +We can achieve the same with the functional approach by providing a function for both cases. + +```cpp +result.and_then([](int& value) { /*do something with the value*/ }) + .or_else([]() { /*handle the case that there is no value*/ }); +``` + +Notice that we get the value by reference, so if a copy is desired it has to be created explicitly in the +[lambda](https://en.wikipedia.org/wiki/Anonymous_function#C++_(since_C++11)) or function we pass. + +The optional can be initialized from a value directly + +```cpp +optional result = 73; +result = 37; +``` + +If the optional is default initialized, it is automatically set to its null value of type `iox::cxx::nullopt_t`. +This can be also done directly by using the constant `iox::cxx::nullopt` + +```cpp +result = iox::cxx::nullopt; +``` + +For a complete list of available functions see +[`optional.hpp`](https://github.com/eclipse-iceoryx/iceoryx/blob/master/iceoryx_hoofs/include/iceoryx_hoofs/cxx/optional.hpp). +The `iox::cxx::optional` behaves like the [`std::optional`](https://en.cppreference.com/w/cpp/utility/optional) +except that it does not throw exceptions and has no undefined behavior. + +## Expected + +`iox::cxx::expected` generalizes `iox::cxx::optional` by admitting a value of another type `E` instead of +no value at all, i.e. it contains either a value of type `T` or `E`. In this way, `expected` is a special case of +the 'either monad'. It is usually used to pass a value of type `T` or an error that may have occurred, i.e. `E` is the +error type. + +For more information on how it is used for error handling see +[error-handling.md](https://github.com/eclipse-iceoryx/iceoryx/blob/master/doc/design/error-handling.md). + +Assume we have `E` as an error type, then we can create a value + +```cpp +iox::cxx::expected result(iox::cxx::success(73)); +``` + +and use the value or handle a potential error + +```cpp +if (!result.has_error()) +{ + auto value = result.value(); + // do something with the value +} +else +{ + auto error = result.get_error(); + // handle the error +} +``` + +If we need an error value, we set + +```cpp +result = iox::cxx::error(errorCode); +``` + +which assumes that `E` can be constructed from an `errorCode`. + +We again can employ a functional approach like this: + +```cpp +auto handleValue = [](int& value) { /*do something with the value*/ }; +auto handleError = [](E& value) { /*handle the error*/ }; +result.and_then(handleValue).or_else(handleError); +``` + +There are more convenience functions such as `value_or` which provides the value or an alternative specified by the +user. These can be found in +[`expected.hpp`](https://github.com/eclipse-iceoryx/iceoryx/blob/master/iceoryx_hoofs/include/iceoryx_hoofs/cxx/expected.hpp). + +Note that when we move an `expected`, the origin contains a moved `T` or `E`, depending on the content before the move. +This mirrors the behavior of moving the content out of the `iox::cxx::expected` like with +`auto foo = std::move(bar.value());` with `bar` being an `iox::cxx::expected`. +Like all objects, `T` and `E` must therefore be in a well defined state after the move. diff --git a/doc/website/design/chunk_header.md b/doc/website/design/chunk_header.md new file mode 100644 index 0000000000..c792cf41ec --- /dev/null +++ b/doc/website/design/chunk_header.md @@ -0,0 +1,287 @@ +# Chunk-Header + +## Summary + +Chunks are the transport capsules in iceoryx. They store data from a publisher as payload and are sent to one or more subscriber. Furthermore, there is some meta information which is stored alongside the payload, e.g. the size and the origin of the chunk. This data is composed in the `ChunkHeader` and located at the front of the chunk. Custom meta information can be added to extend the data from the `ChunkHeader` and tailor the chunk to specific use cases. While this makes the chunk layout more complex, this complexity would otherwise be distributed at different locations all over the code base, e.g. in the request/response feature. Additionally, the adjustments for the user-header makes arbitrary alignments of the user-payload trivial to implement. + +## Terminology + +| Name | Description | +| :---------------- | :------------------------------------------------------- | +| Chunk | a piece of memory | +| Chunk-Header | contains meta information related to the chunk | +| Chunk-Payload | the part of the chunk for the payload which is split into user-header and user-payload | +| User-Header | contains custom meta information, e.g. timestamps | +| User-Payload | the user data with custom alignment | +| Back-Offset | offset stored in front of the user-payload to calculate back to the chunk-header (for the most simple case it will overlap with the user-payload offset stored in the Chunk-Header) | + +Framing with terminology +``` ++=============================================================================+ +| Chunk | ++===================+=========================================================+ +| | Chunk-Payload | +| Chunk-Header +===============+=======+====================+============+ +| | User-Header | ¦ ᶺ | User-Payload | Padding | ++===================+===============+=====|=+====================+============+ + └ Back-Offset +``` + +## Design + +### Considerations + +- it's not uncommon to record chunks for a later replay -> detect incompatibilities on replay +- iceoryx runs on multiple platforms -> endianness of recorded chunks might differ +- for tracing, a chunk should be uniquely identifiable -> store origin and sequence number +- the chunk is located in the shared memory, which will be mapped to arbitrary positions in the address space of various processes -> no absolute pointer are allowed +- in order to reduce complexity, the alignment of the user-header must not exceed the alignment of the `ChunkHeader` + +### Solution + +#### ChunkHeader Definition +``` +class ChunkHeader +{ + uint32_t chunkSize; + uint8_t chunkHeaderVersion; + uint8_t reserved{0}; + uint16_t userHeaderId; + uint64_t originId; + uint64_t sequenceNumber; + uint32_t userHeaderSize{0U}; + uint32_t userPayloadSize{0U}; + uint32_t userPayloadAlignment{1U}; + uint32_t userPayloadOffset; +}; +``` + +- **chunkSize** is the size of the whole chunk +- **chunkHeaderVersion** is used to detect incompatibilities for record&replay functionality +- **reserved** is currently not used and set to `0` +- **userHeaderId** is currently not used and set to `NO_USER_HEADER` +- **originId** is the unique identifier of the publisher the chunk was sent from +- **sequenceNumber** is a serial number for the sent chunks +- **userPayloadSize** is the size of the chunk occupied by the user-header +- **userPayloadSize** is the size of the chunk occupied by the user-payload +- **userPayloadAlignment** is the alignment of the chunk occupied by the user-payload +- **userPayloadOffset** is the offset of the user-payload relative to the begin of the chunk + +#### Framing + +For back calculation from the user-payload pointer to the `ChunkHeader` pointer, the user-payload offset must be accessible from a defined position relative to the user-payload. Lets call this `back-offset`. This is solved by storing the offset in the 4 bytes in front of the user-payload. In case of a simple layout where the `ChunkHeader` is adjacent to the user-payload, this nicely overlaps with the position of `userPayloadOffset` and no memory is wasted. In more complex cases, the offset has to be stored a second time. If the user-payload alignment requires some padding from the header extension, this memory is used to store the offset. + +1. No user-header and user-payload alignment doesn't exceed the `ChunkHeader` alignment + +``` + sizeof(ChunkHeader) userPayloadSize +|------------------>|--------------------->| +| | | ++===================+======================+==================================+ +| Chunk-Header ¦ * | User-Payload | Padding | ++===================+======================+==================================+ +| | | +| userPayloadOffset | | +|------------------>| | +| chunkSize | +|---------------------------------------------------------------------------->| + +*) userPayloadOffset from ChunkHeader and back-offset are overlapping +``` + +2. No user-header and user-payload alignment exceeds the `ChunkHeader` alignment + +``` + sizeof(ChunkHeader) back-offset userPayloadSize +|------------------>| |<---|------------------->| +| | | | | ++===================+=======================+====================+============+ +| Chunk-Header | ¦ | User-Payload | Padding | ++===================+=======================+====================+============+ +| | | +| userPayloadOffset | | +|------------------------------------------>| | +| | +| chunkSize | +|---------------------------------------------------------------------------->| +``` + +Depending on the address of the chunk there is the chance that `ChunkHeader` is the still adjacent to the user-payload. In this case, the framing looks exactly like in case 1. + +3. User-Header is used + +``` + sizeof(ChunkHeader) back-offset userPayloadSize +|------------------>| |<---|------------------->| +| | | | | ++===================+===============+=======+====================+============+ +| Chunk-Header | User-Header | ¦ | User-Payload | Padding | ++===================+===============+=======+====================+============+ +| | | +| userPayloadOffset | | +|------------------------------------------>| | +| | +| chunkSize | +|---------------------------------------------------------------------------->| +``` + +#### User-Payload Offset Calculation + +1. No user-header and user-payload alignment doesn't exceed the `ChunkHeader` alignment + +``` +userPayloadOffset = sizeof(ChunkHeader); +``` + +2. No user-header and user-payload alignment exceeds the `ChunkHeader` alignment, which means the user-payload is either adjacent to the `ChunkHeader` or there is a padding with at least the size of `ChunkHeader` alignment in front of the user-payload and therefore enough space to store the `back-offset` + +``` +chunkHeaderEndAddress = addressof(chunkHeader) + sizeof(chunkHeader); +alignedUserPayloadAddress = align(chunkHeaderEndAddress, userPayloadAlignment); +userPayloadOffset = alignedUserPayloadAddress - addressof(chunkHeader); +``` + +3. User-Header is used + +``` +chunkHeaderEndAddress = addressof(chunkHeader) + sizeof(chunkHeader) + sizeof(userHeader); +anticipatedBackOffsetAddress = align(chunkHeaderEndAddress, alignof(userPayloadOffset)); +unalignedUserPayloadAddress = anticipatedBackOffsetAddress + sizeof(userPayloadOffset); +alignedUserPayloadAddress = align(unalignedUserPayloadAddress, userPayloadAlignment); +userPayloadOffset = alignedUserPayloadAddress - addressof(chunkHeader); +``` + +#### Required Chunk Size Calculation + +In order to fit the user-header and the user-payload into the chunk, a worst case calculation has to be done. We can assume that a chunk is sufficiently aligned for the `ChunkHeader`. + +1. No user-header and user-payload alignment doesn't exceed the `ChunkHeader` alignment + +``` +chunkSize = sizeof(chunkHeader) + userPayloadSize; +``` + +2. No user-header and user-payload alignment exceeds the `ChunkHeader` alignment + +Worst case scenario is when a part of the `ChunkHeader` crosses the user-payload alignment boundary, so that the user-payload must be aligned to the next boundary. The following drawing demonstrates this scenario. + +``` + ┌ back-offset + +===============+==========|+===============================+ + | ChunkHeader | ¦ᵛ| User-Payload | + +===============+===========+===============================+ + +⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- ChunkHeader alignment boundaries +⊥ ⊥ ⊥ ⊥ <- user-payload alignment boundaries + <-----------|---------------> + pre user-payload |-------------------------------> + user-payload alignment user-payload size + alignment + overhang +``` + +The following formula is used to calculate the required chunk size. + +``` +preUserPayloadAlignmentOverhang = sizeof(chunkHeader) - alignof(chunkHeader); +chunkSize = preUserPayloadAlignmentOverhang + userPayloadAlignment + userPayloadSize; +``` + +3. User-Header is used + +Similar to case 2, but in this case it is the `back-offset` which might cross the user-payload alignment boundary. + +``` + ┌ back-offset with same alignment + | as userPayloadOffset + +===============+===========+==============|+===============================+ + | ChunkHeader | UserHeader| ¦ᵛ| User-Payload | + +===============+===========+===============+===============================+ + + ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- userPayloadOffset alignment boundaries +⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- ChunkHeader alignment boundaries +⊥ ⊥ ⊥ ⊥ <- userPayload alignment boundaries + <---------------------------|---------------> + pre user-payload alignment user-payload |-------------------------------> + overhang alignment user-payload size +``` + +The following formula is used to calculate the required chunk size. + +``` +chunkHeaderSize = sizeof(chunkHeader) + sizeof(userHeader) +preUserPayloadAlignmentOverhang = align(chunkHeaderSize, alignof(userPayloadOffset)); +maxPadding = max(sizeof(userPayloadOffset), userPayloadAlignment); +chunkSize = preUserPayloadAlignmentOverhang + maxPadding + userPayloadSize; +``` + +#### Accessing Chunk-Header Extension + +The `ChunkHeader` has a template method to get user-header. There is a risk to use this wrong by accident, but there is currently no better solution for this. + +Since the user-header is always adjacent to the `ChunkHeader`, the formula to get the user-header is: +``` +userHeader = addressOf(chunkHeader) + sizeof(chunkHeader); +``` + +#### ChunkHeader Methods + +- `void* userPayload()` returns a pointer to the user-payload +- `template T* userHeader()` returns a pointer to the user-header +- `static ChunkHeader* fromUserPayload(const void* const userPayload)` returns a pointer to the `ChunkHeader` associated to the user-payload + +#### Integration Into Publisher/Subscriber API + +The `Publisher` has additional template parameters for the user-header and user-payload alignment. Alternatively this could also be done with the `allocate` method, but that increases the risk of using it wrong by accident. + +Additionally, to fully integrate the user-header into the regular control flow, a pointer to a `ChunkHeaderHook` can be passed to the `Publisher` c'tor. + +The `ChunkHeaderHook` is not in the shared memory and has the following virtual methods: +- `virtual void allocateHook(ChunkHeader& chunkHeader)` can be used to initialize the user-header +- `virtual void deliveryHook(ChunkHeader& chunkHeader)` can be used to set members of the user-header, e.g. a timestamp + +Furthermore, the `Publisher` and `Subscriber` have access to the `ChunkHeader` and can use the `userHeader()` method to gain access to the user-header. + +#### Pitfalls & Testing + +- when the user-payload is adjacent to the `ChunkHeader`, it must be ensured that `userPayloadOffset` overlaps with the `back-offset`, which is `sizeof(userPayloadOffset)` in front of the user-payload +- to simplify calculation, it is assumed that the alignment of the user-header doesn't exceed the alignment of the `ChunkHeader`. This has to be enforced with an `assert` + +## Open Issues + +- a `ChunkHeaderHook` could be used on the publisher side +``` +template +class MyChunkHeaderHook : public ChunkHeaderHook +{ + public: + void deliveryHook(ChunkHeader& chunkHeader) override + { + chunkHeader.userHeader().timestamp = myTimeProvider::now(); + } +}; + +auto userHeaderHook = MyChunkHeaderHook(); +auto pub = iox::popo::Publisher(serviceDescription, userHeaderHook); +``` + - alternatively, instead of the ChunkHeaderHook class, the publisher could have a method `registerDeliveryHook(std::function)` + - allow the user only read access to the `ChunkHeader` and write access to the `UserHeader` +- user defined sequence number + - this can probably be done by a `ChunkHeaderHook` + - alternatively, a flag could be introduce into the `ChunkHeader` +- user defined timestamp + - this should probably be in a user-header +- mempool configuration + - currently we specify the chunk-payload size and the size of the `ChunkHeader` is added automatically + - with the new approach, part of the specified chunk-payload size might be used for the user-header + - part of the specified chunk-payload might also be used as padding for the user-payload alignment + - the user will continue to specify the chunk-payload; if a user-header or custom user-payload alignment is used, the user needs to take this into account +- is it necessary to store a flag in `ChunkHeader` if a user extension is used? + - we could maintain a list of known user-header IDs or ranges of IDs similar to `IANA` https://tools.ietf.org/id/draft-cotton-tsvwg-iana-ports-00.html#privateports + - the IDs could be stored in the `ChunkHeader` and everything with an ID larger than `0xC000` is free to use + - to make this somewhat save, the `ChunkHeaderHook` must be mandatory with e.g. a `virtual uint16_t getId() = 0;` method which will be called in the `ChunkSender` + - alternatively, the user-header struct must have a `constexpr uint16_t USER_HEADER_ID`; if it's not present, we could set the ID to `0xC000` which is the first ID free to use +- do we want to store the version of the user extension in the `ChunkHeader`, similarly to `chunkHeaderVersion` +- for record&replay the user-header is totally opaque, which might lead to problems if e.g. a timestamp is stored in the user-header and needs to be updated for the replay + - if we maintain a list of known user-header IDs and also store the user-header version, a record&replay framework could implement the required conversions +- for record&replay it is necessary to store the alignment of the user-payload; decide if a uint16_t should be used of if just the alignment power should be stored in a uint8_t diff --git a/doc/website/design/error-handling.md b/doc/website/design/error-handling.md new file mode 100644 index 0000000000..9f195e782d --- /dev/null +++ b/doc/website/design/error-handling.md @@ -0,0 +1,318 @@ +# Logging and error handling + +## Logging + +Logging is performed by the Iceoryx internal logger. The logger API implements a subset of the Autosar log and trace interface ``ara::log``. +The logger is used internally to record errors and general system runtime information. It can also be used by the user in application code for the same purpose. + +### Logger + +The logger is responsible for logging information about the state of the system to a configurable stream (includes files). + +The logger is thread-safe and can hence be safely used from multiple threads concurrently. Currently the logger is synchronous. + +### Log levels + +The following log levels are supported, ordered by the amount of information displayed from highest to lowest. + +* ``VERBOSE`` - all available information is displayed +* ``DEBUG`` - information to support debugging on developer side +* ``INFO`` - run state information for the user +* ``WARN`` - indicates a potential problem which requires investigation +* ``ERR`` - an error occurred that may be handled on application side or by RouDi +* ``FATAL`` - an error occurred and RouDi is unable to continue +* ``OFF`` - no logging information + +For ``ERR`` and ``FATAL`` see also error levels ``MODERATE``, ``SEVERE`` (logged with ``LogErr``) and ``FATAL`` (logged with ``LogFatal``) in [Error Levels](#Error-Levels). The levels ``ERR`` and ``FATAL`` are only supposed to be used together with the error handler, i.e. need to be accompanied with a corresponding error handler call (currently this cannot be enforced). + +## Error handling + +Errors are considered to be system states that should not be reached regularly and usually are the result of an external failure, such as when the OS is unable to provide a certain resource (e.g. a semaphore) or an application does not respond. In contrast, regular behavior such as a receiver receiving no data when none was sent is not an error. On the other hand, losing data that was sent would be considered an error. + +There are two general approaches to deal with errors: +1. using exceptions + +2. return codes combined with control flow statements and a central instance to handle errors that cannot be mitigated otherwise (the error handler). + +In Iceoryx we use the latter approach. + +### Exceptions +The use of exceptions in Iceoryx is prohibited, including third party code that may throw them. This is due to the following reasons: +* In many implementations exception handling requires dynamic memory (even when the exceptions themselves are not generated dynamically via e.g. new). +* Exception handling is not deterministic with respect to runtime. +* Exception handling may cause a (slight) runtime overhead even when no exceptions are thrown. +* In general it is not possible to incorporate (complete) information about all exceptions that can be thrown in the signature of functions. This makes it hard for the caller to decide whether some exception needs to be handled. +* Overuse of exceptions often leads to convoluted try-catch blocks which makes the code hard to maintain. + +#### Use of noexcept +All functions are marked as ``noexcept``. Note that this does not mean that exceptions cannot be thrown inside such a function. Instead when an exception is thrown inside the function (either directly or indirectly by another function) this exception is not propagated further and ``std::terminate`` will be invoked, calling the terminate handler. The goal is that this cannot happen under any circumstances during runtime. + +To this end it is necessary to eliminate the use of all potentially throwing (third party) functions throughout the codebase. Since many STL functions may throw, these cannot be used either and their functionality needs to be reimplemented without the use of exceptions. In particular anything allocating dynamic memory may throw a ``std::bad_alloc`` exception when memory is exhausted. + +#### Alternatives to exceptions +As an alternative to exceptions we use the error handler and a variation of return codes in the form of ``cxx::expected``, described below. ``cxx::expected`` can be used to communicate the error to the caller, who has to decide whether to handle the error itself or propagate it further (e.g. as another ``cxx::expected``). Error handling itself is performed by the error handler which handles errors occurring in the subcomponents of ``iceoryx::posh``. + +### Error Handler +The error handler is called internally when an error is detected in the Iceoryx middleware daemon (RouDi) or the Iceoryx runtime. The error handler should only be called in exceptional situations (invalid access errors, out of resources etc.) and not in circumstances that occur regularly (it is sort of an exception replacement). + +If the exceptional situation can be resolved without calling the error handler (e.g. by delegating an appropriate return value to the caller), this should be preferred (since the error handler is a last resort mechanism). + +It is not supposed to be called by applications at any time. + +### Technical Requirements +* The error handler must be reentrant. +* The error handler must be thread-safe. +* The error handler uses the logger but the logger cannot depend on the error handler. +* When a fatal error is detected and termination is required, the reporting thread shuts down the RouDi gracefully. +* A custom error handler function can be installed (e.g. for testing). +* When the reaction on an error is just logging, computation in other threads shall not be influenced. +* If the error does not require termination, the error handler must return eventually. + +### Error Levels +The following error levels are supported. + +* ``MODERATE`` +* ``SEVERE`` +* ``FATAL`` + +#### MODERATE +A recoverable error. Leads to an error log entry (``LogErr``) and continues execution. + +**Example:** +1) RouDi receives an unexpected message and discards it. The remaining communication proceeds normally. +2) A port requested by an application cannot be provided due to e.g. resource exhaustion. + +#### SEVERE +RouDi may continue but applications may be compromised or the functionality reduced. Leads to an error log entry (``LogErr``) and assert, terminating execution in debug mode. The handler must return to be able to continue execution in release mode. + +**Example:** +A message queue is overflowing and messages are lost. RouDi can continue but lost data may affect applications. + +#### FATAL +RouDi cannot continue and will shut down. Leads to an error log entry (``LogFatal``), assert and calls ``std::terminate``, terminating execution in debug and release mode. +Before calling terminate, a callback is invoked (if configured), which can execute specific error handling code (e.g. call a 3rd party error handler). +The handler is not required to return here (since this may not be always possible or reasonable). The reporting code should still try to proceed to a safe state if possible in order to improve testability in case of such errors. + +A fatal error in the runtime terminates the application. + +**Example:** +RouDi is unable to allocate sufficient shared memory. + +### Error Codes and Additional Information + +Currently error codes are used to identify the location of an error. These are provided as an enum in error_handling.hpp. To uniquely identify an error location, these have to be different for each error. While this does not necessarily have to be the case, it is currently advised as there is no additional information (i.e. line, file) yet which allows to distinguish between different errors with the same code. + +In addition a user callback may be provided. Currently it cannot take arguments directly but since passing a generic callable with signature ``void(void)`` is possible, a lambda or functor object containing additional arguments can be provided. + +### Expects and Ensures + +These assert-like constructs are used to document assumptions in the code which are checked (at least) in debug mode. Currently they are always active, i.e. also checked in release mode. If the condition is violated they print the condition, the location of occurrence in the code and terminate the program execution. + +Since they are not necessarily active in release mode, they cannot be used to handle errors. Their purpose is to detect misuse or bugs of the API early in debug mode or to verify a result of an algorithm before returning. In this way, assumptions of the developer are made explicit without causing overhead when not needed. Therefore errors to be caught by Expects and Ensures are considered bugs and need to be fixed or the underlying assumptions and algorithms changed. This is in contrast to errors which are expected to occur during runtime which are handled by the error handler (i.e. a system resource cannot be obtained). + +Although Expects end Ensures behave the same, the former is used to signify a precondition (e.g. arguments of a function) is checked, while the latter indicates a postcondition check (e.g. result of a function before returning) + +Examples include expecting pointers that are not null (as input, intermediate or final result) or range checks of variables. + +### cxx::expected + +``cxx::expected`` is a template which either holds the result of the computation of type ``T`` or an object of error type ``E``. The latter can be used to obtain additional information about the error, e.g. an error code. +In a way this extends error codes and may act as kind of a replacement of exceptions. It is usually used as return type of functions which may fail for various reasons and should be used if the error is supposed to be delegated to the caller and handled in the caller context. +It is also possible to further propagate the error to the next function in the call-stack (since this must be done explicitly, which is comparable to rethrowing an exception). + +It is possible to use the ``nodiscard`` option to force the user to handle the returned ``cxx::expected``. +If the error cannot be handled at a higher level by the caller, the error handler needs to be used. + +Examples include wrapping third party API functions that return error codes or obtaining a value from a container when this can fail for some reason (e.g. container is empty). If no additional information about the error is available or required, ``cxx::optional`` can be used instead. + +### Error Handling in posh + +Error logging shall be done by the logger only, no calls to ``std::cerr`` or similar should be performed. + +All the methods presented (``cxx::expected``, ``Expects`` and ``Ensures`` and the error handler) can be used in posh. The appropriate way depends on the type of error scenario (cf. the respective sections for examples). The error handler should be considered the last option. + +### Error Handling in hoofs + +Error logging is currently done by calls to ``std::cerr``. In the future those might be redirected to the logger. + +The error handler cannot be used in hoofs. + +Whether it is appropriate to use ``cxx::expected`` even if STL compatibility is broken by doing so depends on the circumstances and needs to be decided on a case-by-case basis. If the function has no STL counterpart ``cxx::expected`` can be used freely to communicate potential failure to the caller. + +It should be noted that since currently `Expects` and `Ensures` are active at release mode, prolific usage of these will incur a runtime cost. Since this is likely to change in the future, it is still advised to use them to document the developer's intentions. + +Do not use `std::terminate` directly but use assert-like constructs such as ``Expects`` or ``Ensures``. + +### Interface for 3rd Party Code + +The Error handler as well as logger shall be able to use or redirect to 3rd party error handling or logging libraries in the future. Currently this is neither fully supported nor used. The error handler has a callback function which can in principle be used to call 3rd party code. + +## Usage + +### Logger +The logger can be used similar to the streams in the C++ standard API. +To select the log level, the corresponding logger has to be used, e.g. ``LogErr``, ``LogWarn`` etc. + +``` +LogWarn() << "log message " << someValue << "log message continued"; +``` +A line break is inserted implicitly at the end (after "log message continued" in the example). + +### Error Handler +The most general use case is the following +``` +if(noError()) { + //handle the regular case +} else { + auto callback = []() { //some error handling callback}; + errorHandler(Error::kSOME_ERROR_CODE, callback, ErrorLevel::SEVERE); +} +``` + +If no callback or error level are specified, the error level is assumed to be FATAL by default. +``` +errorHandler(Error::kSOME_ERROR_CODE) +``` + +If no callback but an error level is desired, a *nullptr* has to be provided for the callback. +``` +errorHandler(Error::kSOME_ERROR_CODE, nullptr, ErrorLevel::MODERATE); +``` + +### Expects and Ensures + +Assume myAlgorithm is part of an inner API and not supposed to be called with a ``nullptr``. We may have used a reference here, this is just for illustration. +In addition the value pointed to is assumed to be in the range (-1024, 1024). While we could check this every time, this can be avoided if we specify that the caller is responsible to ensure that these conditions hold. + +``` +int32_t myAlgorithm(int32_t* ptr) { + Expects(ptr!=nullptr); + //observe the order, we only dereference after the nullptr check + Expects(*ptr > -1024 && *ptr < 1024); + + int32_t intermediate = timesTwo(*ptr); + //this may not be necessary here to ensure that the next function call is valid, + //but it states our expectations clearly + Ensures(intermediate % 2 == 0); + + int32_t result = abs(intermediate); + + Ensures(result % 2 == 0); + Ensures(result >= 0); + Ensures(result < 2048); + + return result; +} +``` +Note that in the case of ``nullptr`` checks it is also an option to use references in arguments (or ``not_null`` if it is supposed to be stored since references are not copyable). It should be considered that ``not_null`` incurs a runtime cost, which may be undesirable. +When Expects and Ensures are implemented to leave no trace in release mode, we do not incur a runtime cost using them. For this reason, it is advised to use them to document and verify assumptions where appropriate. + +### cxx::expected +This example checks the arguments and if they are valid proceeds to compute a result and returns it. +Otherwise it creates an Error object from an errorCode and returns it. + +``` +cxx::expected func(Arg arg) { + int32_t errorCode = checkArg(arg); + if(isNoError(errorCode)) { + SomeType result = computeResult(arg); + // optionally do something with result + return result; + } + return Error(errorCode); +} +``` +The caller is responsible for handling (or propagating) the error. + +``` +auto result = func(arg); +if(result.has_error()) { + auto& error = result.error(); + //handle or propagate the error +} else { + auto& value = result.value(); + //proceed by using the value +} +``` + +Alternatively a functional approach can be used. + +``` +auto successFunc = [](SomeType& value) { + //proceed by using the value +}; + +auto errorFunc = [](Error& error) { + //handle the error +}; + +func(arg).and_then(successFunc).or_else(errorFunc); +``` + +## Open Points + +### Centralized Error Handling + +It may be desirable to have a centralized error handling instance where runtime errors on application side are logged and (maybe) handled. +This could also be done in RouDi (by sending information to RouDi), but RouDi already has too much responsibility. Preferably this should be done by a separate application with this sole purpose. +If the application cannot reach the central handler, it shall try to handle the error locally if possible (at least log it). + +However, it might be too slow if this would rely on error transmission and responses. If this is to be implemented, the exact mechanism has to be decided on. + +### 3rd Party Error Handling +We need to decide how to provide an interface for 3rd party error handling, especially for the runtime. This interface will probably rely on hooks/callbacks. The signature and callsites of these need to be discussed. +This is related to centralized error handling as well. + +### Overriding Specific Error Reaction +* Do we want to provide the ability to override error reaction based on e.g. error codes? +* Do we want to disable certain error levels? (if so, this should preferably happen at compile +time with no or few runtime artifacts). It could be an option to e.g. disable all ``MODERATE`` error reaction at compile time. +* It is probably not reasonable to allow disabling reaction on fatal errors. + +This is also related to the hooks for 3rd party error handling we may want to provide. + +### Return in Case of Fatal Error +The reporting code does not need to be able to continue properly in case of a fatal error, but there needs to be a return after the error handler call. While the error handler is not required to return, it still might under certain circumstances (e.g. a mock error handler in a test case). + +The (complete) intended behavior of the error handler requires some further clarification, especially in the case of fatal errors. In the case of non-fatal errors the code invoking the error handler must be able to continue after the error handler returns. + +### Error Handling vs. Logging +Does it make sense to have ``LogErr`` without an error handler call? If an error occurs it should probably be enforced that the handler is called and not just lead to a log entry. +One reason for this is that currently it is not possible to provide an additional error message to the error handler. + +### Additional Error Information +It would be desirable to allow the possibility to provide additional messages (or even general functions/arguments) to the error handler, which is currently missing. +This can be combined with the addition of error location to the error handler. + +An optional stack-trace (at least in debug mode) may also prove very useful. +What is needed to have a limited stack-trace even in release mode? + +### Debug vs. release mode +We need to further clarify behavior in release and debug mode of the error handler and ``Expects`` and ``Ensures`` (and maybe the logger as well). Can we have a release build with additional information? (e.g. symbols for a stack-trace). + +### Assert +Do we want an Assert in addition to ``Expects`` and ``Ensures``? If so, shall it possibly be active in release mode or only debug mode? + +In principle with a sufficiently powerful Assert or ``Expects`` (resp. ``Ensures``), this should not be needed (they are equivalent in their functionality). + +## Future improvements +In this section we briefly describe ways to potentially improve or extend functionality of existing error handling components. This list is by no means exhaustive and all suggestions are up for discussion and may be refined further. + +### Logger +1. The logger could be extended to include logging over a network to a remote server or similar. +2. Support asynchronous logging. + +### Error Handler +1. Allow customization for ``MODERATE`` and ``SEVERE`` errors to continue according to a user defined configuration. +2. Add file, line and function information (using ``__FILE__``, ``__LINE__`` and ``__func__``). This would require using macros for the error handler call in a future implementation. +3. Allow generalized callbacks with variadic arguments. +4. Change the order of arguments in a future design (callback and additional arguments last). Providing the callback and potential arguments can be made fully optional this way. +5. If deactivation or reduced operation (e.g. not handling ``MODERATE`` errors) is desired, this partial deactivation should cause no (or at least very little) runtime overhead in the deactivated cases. + +### Expects and Ensures +Allow deactivation in release mode, but it should still be possible to leave them active in release mode as well if desired. Deactivation in debug mode can also be considered but is less critical. Deactivation should eliminate all runtime overhead (i.e. condition evaluation). + +### cxx::expected +1. Consider renaming ``cxx::expected`` to ``cxx::result``, which is more in line with languages such as Rust and conveys the meaning more clearly. +2. Add ``has_value`` (in addition to ``has_error``) for consistency with ``cxx::optional``. +3. Improve the monadic error handling (beyond ``and_then``, ``or_else``) to allow for better pipelining of multiple consecutive calls (especially in the success case). This requires careful consideration of supported use cases and intended behavior but can reduce control flow code (``if ... else ...``) in error cases considerably. diff --git a/doc/website/design/listener.md b/doc/website/design/listener.md new file mode 100644 index 0000000000..f8099fd7a2 --- /dev/null +++ b/doc/website/design/listener.md @@ -0,0 +1,317 @@ +# Listener + +## Summary + +The Listener is besides the WaitSet one of the building blocks to realize a +push approach to detect and react to certain events. The Listener +offers the user the ability to attach objects with a corresponding event and +callback. Whenever the object receives the specified event the corresponding +callback is called in the Listener background thread as a reaction. + +The two key differences to the WaitSet are that the Listener is event-driven +and not event- and state-driven as the WaitSet and that the Listener creates +a separate background thread in which the event-callbacks are executed in contrast +to the WaitSet where the user has to call the event-callbacks explicitly. + +## Terminology + +- [Condition Variable](https://en.wikipedia.org/wiki/Monitor_(synchronization)#Condition_variables_2) + Used by an attached object to inform the Listener/WaitSet that an event has + occurred. +- **event** is changing the state of an object. +- **event driven** a one time reaction which is caused directly by an event. + Example: a new sample has been delivered to a subscriber. +- **state** predefined values to which the members of an object are set. +- **state driven** a repeating reaction which is continued as long as the state + persists. + Example: a subscriber has stored samples which were not inspected by the user. + +## Design + +The Listener is a variation of the +[reactor pattern](https://en.wikipedia.org/wiki/Reactor_pattern) and +the usage should be similar to the _WaitSet_ with a key difference - it should +be **event driven** and not a mixture of event and state driven, depending on +which event is attached, like in the _WaitSet_. + +### Requirements + +- Whenever an event occurs the corresponding callback should be called **once** + as soon as possible. +- If an event occurs multiple times before the callback was called, the callback + should be called **once**. +- If an event occurs while the callback is being executed the callback should be + called again **once**. +- Thread-safety: attaching an event at any time from any thread. +- Thread-safety: detaching an event at any time from any thread. + - If the callback is currently running `detachEvent` blocks until the callback + is finished. + - After the `detachEvent` call the event-callback is not called anymore, even + when the event was signalled while `detachEvent` was running and the + callback was not yet executed. + - If the callback is detached from within the event-callback then `detachEvent` + is non blocking. The event is detached right after the `detachEvent` call. +- One can attach at most one callback to a specific event of a specific object. + - Usually defined with an enum by the developer. One example is `SubscriberEvent::DATA_RECEIVED`. + - Attaching a callback to an event where a callback has been already attached + results in an error. +- One can attach the same event to different objects at the same time. +- One can attach multiple different events to a single object +- When the Listener goes out of scope it detaches itself from every attached + object via a callback provided by the attached object (like in the WaitSet). +- When the class which is attached to the Listener goes out of scope it + detaches itself from the Listener via a callback provided by the Listener + (like in the WaitSet). + +### Solution + +#### Class diagram + +``` + +---------------------------+ + | ConditionVariableData | + | - m_semaphore | + | - m_runtimeName | + | - m_toBeDestroyed | + | - m_activeNotifications | + +---------------------------+ + | 1 | 1 + | | + | 1 | n ++-----------------------------------------------+ +--------------------------------------------------+ +| ConditionListener | | ConditionNotifier | +| ConditionListener(ConditionVariableData & ) | | ConditionNotifier(ConditionVariableData &, | +| | | uint64_t notificationIndex) | +| bool wasNotified() | | | +| void destroy() | | void notify() | +| NotificationVector_t wait() | | | +| NotificationVector_t timedWait() | | - m_condVarDataPtr : ConditionVariableData* | +| | | - m_notificationIndex | +| - m_condVarDataPtr : ConditionVariableData* | +--------------------------------------------------+ +| - m_toBeDestroyed : std::atomic_bool | | 1 ++-----------------------------------------------+ | + | 1 | n + | +--------------------------------+ + | 1 | TriggerHandle | ++-------------------------------------------------+ | bool isValid() | +| Listener | | bool wasTriggered() | +| attachEvent(Triggerable, EventType, Callback) | | void trigger() | +| detachEvent(Triggerable, EventType) | | void reset() | +| | | void invalidate() | +| - m_events : Event_t[] | | void getUniqueId() | +| - m_thread : std::thread | | | +| - m_conditionListener : ConditionListener | | - m_conditionVariableDataPtr | +| | | - m_resetCallback | +| +----------------------------+ | | - m_uniqueTriggerId | +| | Event_t | | +--------------------------------+ +| | void executeCallback() | | | 1 +| | bool reset() | | | +| | bool init(...) | | | n +| | | | +-------------------------------------------------------+ +| | - m_origin | | | Triggerable (e.g. Subscriber) | +| | - m_callback | | | | +| | - m_invalidationCallback | | | void invalidateTrigger(const uint64_t triggerId) | +| | - m_eventId | | | void enableEvent(TriggerHandle&&, const EventEnum ) | +| +----------------------------+ | | void enableEvent(TriggerHandle&&) | ++-------------------------------------------------+ | void disableEvent(const EventEnum) | + | void disableEvent() | + | | + | - m_triggerHandle : TriggerHandle | + +-------------------------------------------------------+ +``` + +The Triggerable does not need to implement all `enableEvent`, +`disableEvent` variations, only the ones which are required by the use +case. The `enableEvent` and `disableEvent`, without the distincting `EventEnum`, +can be used when there is only a single event which can be triggered. + +#### Class Interactions + +- **Creating Listener:** a `ConditionVariableData` is created in the shared memory + The `Listener` uses the `ConditionListener` to wait for incoming events. + +``` + PoshRuntime +Listener | + | getMiddlewareConditionVariable : var | + | --------------------------------------> | + | ConditionListener(var) | ConditionListener + | ----------------------------------------+-----------------> | + | wait() : vector | | + | ----------------------------------------+-----------------> | +``` + +- **Attaching Triggerable Event (SubscriberEvent::DATA_RECEIVED) to Listener:** + The Listener creates a TriggerHandle and provides it to the Triggerable (Subscriber) + via `enableEvent` so that the Triggerable owns the handle. Whenever the event occurs + the Triggerable can use the `trigger()` method of the TriggerHandle to notify + the Listener. + +``` +User Listener Triggerable + | attachEvent() | | + | ------------------> | TriggerHandle | + | | create | | + | | ---------> | | + | | enableEvent(std::move(TriggerHandle)) | + | | -----------+--------------------------------------> | +``` + +- **Signal an event from Triggerable:** `TriggerHandle::trigger()` is called, the + Listener is returning from the `ConditionListener.wait()` call and retrieves + a list of all the signal notifications. The corresponding event-callbacks + are called. + +``` +Triggerable TriggerHandle ConditionNotifier ConditionListener Listener Event_t + | trigger() | | | wait() : notificationIds | | + | -------------> | notify() | | <------------------------- | | + | | -------------> | .... unblocks .... | blocks | exeuteCallback() | + | | | | | ------------------------> | + | | | | | m_events[notificationId] | +``` + +- **Triggerable goes out of scope:** The TriggerHandle is a member of the + Triggerable, therefore the d'tor of the TriggerHandle is called which removes + the trigger from the Listener via the `resetCallback` + +``` +Triggerable TriggerHandle Listener Event_t + | ~TriggerHandle | | | + | ----------------> | removeTrigger() | | + | | ----------------> | reset() | + | | via resetCallback | ------------> | +``` + +- **Listener goes out of scope:** The d'tor of the `Event_t` invalidates the + Trigger inside the Triggerable via the `invalidationCallback` + +``` +Listener Event_t Triggerable + | ~Event_t() | | + | -----------> | invalidateTrigger() | + | | -----------------------> | + | | via invalidationCallback | +``` + +#### TriggerHandle + +- **Problem:** The Triggerable should be able to notify a Listener/WaitSet + without having any knowledge about those class so that circular + dependencies can be prevented. Furthermore, the Triggerable must be able + to remove its attached events when it goes out of scope. +- **Solution:** The dependency inversion principle, create an abstraction which + is known by both, the TriggerHandle. Created by the Listener/WaitSet and + attached to the Triggerable so that it can notify the Listener/WaitSet via + the underlying `ConditionNotifier` with `TriggerHandle::notify()`. + The cleanup task is performed by the `m_resetCallback` so that the + Triggerable has no dependencies to any Notifyable. + +#### Condition Variable + +The `ConditionListener` and `ConditionNotifier` are two different interfaces to +the same class which state is stored in the `ConditionVariableData` class. The +intention of the separation is to provide one side (e.g. Triggerable) only +an API to notify the Notifyable (e.g. Listener) whereas the Notifyable can only +wait on events. So the contract is reflected in the design. + +- **Problem:** Since the Listener reacts on events and not states it requires + the knowledge by whom it was notified. +- **Solution:** + - Every TriggerHandle has a unique id which is used as index in `ConditionNotifier`. + - When `ConditionNotifier::notify` is called the Listener is informed via the + `NotificationVector_t` return value from `ConditionListener::wait()` which + index notified him. Hence it is the same as the unique id of the TriggerHandle + the Listener knows which Triggerable notified him. + +#### Event_t + +##### Concurrency + +The Listener must be able to attach, detach events concurrently. Additionally, +it supports that a callback can attach or detach further events or detach its +corresponding event concurrently. Furthermore, the Listener supports that a +callback is called concurrently while events are being attached/detached. + +To realize this we created the `Event_t` abstraction which is stored in an +array called `m_events`. If we would like to attach or detach an event we either +initialize `Event_t::init()` or reset `Event_t::reset()` the corresponding entry +in the `m_events` array. The array has the advantage that the data structure +itself never changes during runtime therefore it does not have to be thread-safe. + +The thread-safety must then be ensured by the `Event_t` class itself. Since +every concurrent action is contained in `Event_t` we can use +`concurrent::smart_lock` in combination with a `std::recursive_mutex` to +guarantee the thread-safe access. + +1. Concurrent attach/detach event and callback execution, ensured by + thread-safe `Event_t`. +2. Detaching itself from within a callback is ensured via the + `std::recursive_mutex`. +3. Attaching/detaching arbitrary events from within a callback is ensured by + securing every `Event_t` object in `m_events` with a `concurrent::smart_lock`. + This would not be possible if the data structure itself had to be thread-safe. + +##### Lifetime + +Since the event contains everything which is required to handle events it has +the responsibility to ensure the life-time of the TriggerHandle. This is done +by the `m_invalidationCallback` which is called in `Event_t::reset()` to +invalidate the TriggerHandle in the corresponding Triggerable. This is either +done when an event is detached or the Listener goes out of scope. + +#### Triggerable (e.g. Subscriber) + +The Triggerable is a set of classes of which events can be attached to the Listener. + +It is possible to either attach a specific event of a class to the Listener or +attach the class without providing an event. + +The basic idea is that the Listener creates a TriggerHandle whenever an event +is attached and provides that TriggerHandle to the corresponding Triggerable. +The Triggerable uses then the TriggerHandle to notify the Listener about events. + +##### Triggerable With Single Event + +Every Triggerable requires: + +1. The private methods: + +```cpp +void enableEvent(iox::popo::TriggerHandle&& triggerHandle) noexcept; +void disableEvent() noexcept; +void invalidateTrigger(const uint64_t uniqueTriggerId) noexcept; +``` + +##### Triggerable With Multiple Events + +Every Triggerable requires: + +1. An `enum class` which uses `iox::popo::EventEnumIdentifier` as underlying type. + +```cpp +enum class EventEnum : iox::popo::EventEnumIdentifier { + EVENT_IDENTIFIER, + ANOTHER_EVENT_IDENTIFIER, +}; +``` + +2. The private methods: + +```cpp +void enableEvent(iox::popo::TriggerHandle&& triggerHandle, const EventEnum event) noexcept; +void disableEvent(const EventEnum event) noexcept; +void invalidateTrigger(const uint64_t uniqueTriggerId) noexcept; +``` + +The methods above are used by the Listener to transfer the ownership of the TriggerHandle +to the Triggerable. The Triggerable should have one TriggerHandle member for every +attachable event/state. +The TriggerHandle is then used to notify the Listener by the Triggerable that +a certain event has occurred. + +3. It must be friend with `iox::popo::NotificationAttorney`. It is possible to + provide public access to the previous methods but then the user has the ability + to call methods which should only used by the Listener. + + diff --git a/doc/website/design/lockfree_queue.md b/doc/website/design/lockfree_queue.md new file mode 100755 index 0000000000..251165565d --- /dev/null +++ b/doc/website/design/lockfree_queue.md @@ -0,0 +1,135 @@ +Remark: This is a preliminary description and will be reworked when the capacity feature is implemented. + +# Lock Free queue + +We explain some details of the lock free queue in order to provide an intuition for the way the lock free queue works. This could serve as a basis for a formal proof later if desired. + +## Index queue analysis + +In the following all numbers are natural numbers and hence bounded unsigned integers in the implementation. + +### Capacity and Cycle Length +The index queue has some capacity n and stores indices in the range [0, n[. +The number n is also called cycle length. + +Assume n = 4 in this explanation, so we can store the indices 0, 1, 2, 3. + +### Index representation +Each index i corresponds to an equivalence class [i] modulo n, i.e. +[i] = {j | j = c*n + i, c >= 0} + +This means j = 3, 7, 11, ... all represent the same index 3 (pointing to the last element in the array) + +Given j, we have i = j % n and c = j / n (where % is modulo and / integer division). + +Another way to represent such a value j uniquely is as a pair (c,i) and +we call this a cyclic index with cycle c and index i. + +This is done for two reasons: detection of empty queues and to eliminate the ABA problem for all practical scenarios. +It is imperative that such indices can be used in compare and swap operations (CAS), i.e. +will not exceed 64 bits on standard architectures supporting 64 bit CAS. +Therefore j can be assumed to be an unsigned 64 bit integer. + +### Queue representation +The queue has a head H, tail T and an array of n values. All of them are cyclic indices as in 2. +Initially the queue is empty and head and tail point to index 0 but have both cycle 1. +Array elements are 0 and represented as (0,0). + +Initially empty queue +``` +[ (0,0), (0,0), (0,0), (0,0) ] H=(1,0) T=(1,0) +``` +Initially full queue with all indices +``` +[ (0,0), (0,1), (0,2), (0,3) ] H=(0,0) T=(1,0) +``` + + +### Head and Tail monotonicity +We always insert (push) at Tail and remove (pop) at Head. +Head and Tail both increase strictly monotonic (making ABA problems unlikely). +I.e. a push causes tail to increase by one (causing the cycle to increase after n pushes) and similarly each pop causes Head to increase by one. + +### Push operation +push(y) causes the cycle at the position Tail points to to be replaced with Tails cycle while the element there is replaced with x +via a CAS operation. +Afterwards Tail is increased by 1 (potentially not immediately, but before the next push takes effect). +``` +[ (c,?), (c,x), (c-1,?), (c-1,?) ] H=(c,1) T=(c,2) +``` +push(y) +``` +[ (c,?), (c,x), (c,y), (c-1,?) ] H=(c,1) T=(c,3) +``` + +We only push if the cycle at the element Tail points to is exactly one behind Tails cycle. + +Constraint: we can never push more than n elements (and our use case does not require this). + +### Pop operation +Pop reads the value at the index of head and returns it if the cycle matches Heads cycle and a CAS to increase head by 1 succeeds. + +``` +[ (c,?), (c,y), (c,x), (c-1,?) ] H=(c,1) T=(c,3) +``` +pop returns y +``` +[ (c,?), (c,x), (c,y), (c-1,?) ] H=(c,2) T=(c,3) +``` + +If the cycle is behind Heads cycle the queue is empty (at the time of this check) and nothing is returned. + + +### Head Tail relationship +Head is always at most Tail, i.e. H <= T. (Technically we only have H - 1 <= T in general, cf. push implementation) + +The general situation is one of the following, with c indicating the cycle of the values and Head and Tail. + +(i) Head and Tail are at cycle c and ***** marks the region in the array with values logically in the queue. +``` +[ c |*****c*****| c-1 ] + H = (c,i) T= (c,j) +``` + +(ii) Head is one cycle behind Tail. +``` +[******c*****| c-1 |******c-1*****] + T = (c,i) H= (c-1,j) +``` + +(iii) Empty queue +``` +[ c | c-1 ] + T = H = (c,i) +``` +### ABA prevention + +Note that the monotonicity of Head and Tail combined with suitable CAS make ABA problems not a practical concern. +Moreover, even if we require many bits for the index and thus have few bits for the cycle, we still require a complete uint64 +wraparound for an ABA problem to occur (in addition to re-inserting the same value). + +This is simply because if the maximum cycle is small then the cycle length n must be large and +we increase the cycle every n pushes (or pops), but the numbers representing the index increase by one every operation. + +### Lock free analysis +Claim: +The queue is operation-wise lock free: pushes cannot block pops arbitrarily and vice versa. +Furthermore, between concurrent pushes and pops one of each type always succeeds in a finite amount of time. + +Proof sketch: +- only pushes read/write Tail but pops do not +- pushes modify array values which pop reads and compares (CAS) + but we can have at most n pushes before this stops (push constraint in 5.) and then pop will succeed without interference from push +- only pops read/write/CAS Head +- claim follows now from loop/read/write/CAS structure in implementation + +Therefore pushes and pops do not interfere to block each other. + +However, pushes can block other pushes and pops other pops. +In particular operation-wise lock freedom implies the queue is lock free. + +Note that there is no fairness guarantee. In principle the same push thread or pop thread might always succeed but in practice this is unlikely. +The queue is therefore not wait free. + +This could technically be mitigated somewhat with a more complex logic keeping track of operation failures +but seems not to be worth it in practice. (incomplete analysis, further studies required) diff --git a/doc/website/design/relocatale_pointer.md b/doc/website/design/relocatale_pointer.md new file mode 100644 index 0000000000..833311b4f0 --- /dev/null +++ b/doc/website/design/relocatale_pointer.md @@ -0,0 +1,215 @@ +# Relocatable and Relative Pointers + +When memory is shared between applications and each application maps the shared memory to a different starting address +in its own address space, raw pointers cannot be used anymore. This is due to the fact that a pointer pointing to some +object in shared memory in one address space will in general not point to the same object in another address space (or +even to the shared memory), i.e. the pointer is invalid. In this case, different pointer concepts such as relative and +relocatable pointers must be used. Depending on the shared memory setup and the relationship between pointer and pointee, +there are different use cases which we need to cover. + +Both RelativePointer and RelocatablePointer are template pointer classes that try to retain much of the normal pointer +syntax where possible while still being valid across address space boundaries. This means that operations like +assignment, dereferencing, comparison and so on work similar to raw pointers. They do not possess reference counting +semantics in this implementation, but extensions for this functionality are possible. + +## Shared Memory Setup + +In general, we have many shared memory segments, each starting at some address and containing a specified number of +bytes. Specifically we distinguish between management and payload segments. The management segment contains metadata +used by RouDi and the runtime to communicate, store common structures such as ports and so on. The payload segment +contains the actual data samples sent by senders in some application to receivers in other applications. + +## Relocatable pointer + +When pointer and pointee are located in the same shared memory segment, the lightweight (compared to a relative pointer) +relocatable pointer can be used. That is, we have the following scenario. + +Pointer p points to object X of type T and both are stored in shared memory segment S. + +``` +Shared Memory S: p X + |____________________^ + +App1 a1 b1 c1 +App2 a2 b2 c2 +``` + +Let a1, b1, c1 and so on be the addresses of segment S, pointer p and object X in application 1 and similarly a2, b2 and +c2 in application2. If application 2 maps the memory differently they will be shifted by some common offset d depending +on the individual memory mapping: + +a2 = a1 + d, b2 = b1 + d, c2 = c1 + d + +This is why storing a raw pointer to X will not be sufficient, the value c1 of p will not point to X in application 2. +However, storing the difference between the location of p and X will work since it is an invariant in both address +spaces. + +Thus, instead of using a raw pointer p, we just use a relocatable pointer RelocatablePointer pointing to X. This pointer +can then be dereferenced in both address spaces and will point to X. + +Note that if we only use relocatable pointers in this way in complex data structures that rely on pointers (such as +lists and trees) in shared memory, we can in fact copy the whole shared memory segment somewhere else (i.e. *relocate* +it, hence the name) without compromising the data structure integrity. That is, the list elements in the copy correctly +point to their successor elements in the copied memory. This also presupposes that we do not rely on other side effects +of deep copying (i.e. nontrivial copy constructors). + +### Construction and Assignment + +Assume we have an object X of type T in shared memory. Then we can construct relocatable pointers p in the *same +segment* (e.g. as members of objects that reside in this segment or via emplacement new at a memory address in this +segment). + +default construct a logical nullptr ++ `RelocatablePointer p;` + +construct relocatable pointing to X ++ `RelocatablePointer p(&X);` + +assign relocatable to point to X ++ `p = &X;` + +### Operations +Most operations for raw pointers also work for relocatable pointers. In particular, a RelocatablePointer is compatible +with a raw pointer T* in many cases, such as assignment, to provide seamless integration and convenient syntax. + +comparison with raw pointers ++ `if(p != nullptr) { \\do something }` + ++ `T* raw = &X; if(p == raw) { \\do something }` + +assignment to raw pointer (resolves the pointer in the current address space) ++ `T* raw = p;` + +explicitly get the raw pointer ++ `T* raw = p.get();` + +dereference ++ `T& t = *p;` + +access members ++ `p->func();` + +### Performance + +Since relocatable pointers just measure the distance to the pointee from itself it just requires a number to store this +pointer difference. This number has the same size as a pointer itself, i.e. 64 bit on 64 bit architectures. All +operations use just a few simple arithmetic instructions in addition to pointer dereferencing and hence they do incur +little overhead compared to raw pointers. + +## Relative pointer + +When pointer and pointee are located in different shared memory segments, it gets more complicated and relocatable +pointers are no longer appropriate. This happens in our setup with metadata in the management segment referencing data +in the payload segment. + +Pointer rp is stored in segment S1 and points to object X of type T in segment S2. + +``` +Shared Memory S1: rp S2: X + |_______________________^ + +App1 a1 b1 c1 d1 +App2 a2 b2 c2 d2 +``` + +Now it is no longer true in general that both segments will be offset by the same difference in App2 and therefore +relocatable pointers are no longer sufficient. + +Relative pointers solve this problem by incorporating the information from where they need to measure differences (i.e. +*relative* to the given address). This requires an additional registration mechanism to be used by all applications +where the start addresses and the size of all segments to be used are registered. Since these start address may differ +between applications, each segment is identified by a unique id, which can be provided upon registration by the first +application. In the figure, this means that the starting addresses of both segments (a1, a2 and c1, c2) would have to be +registered in both applications. + +Once this registration is done, relative pointers can be constructed from raw pointers similar to relocatable pointers. +However, it should be noted that relocating a memory segment will invalidate relative pointers, i.e. relative pointers +are NOT relocatable. This is because the registration mechanism cannot be automatically informed about the +copy of a whole segment, such a segment would have to be registered on its own (and the original segment deregistered). + +### Registration + +register the start pointer start1 of a segment with a specific size (this happens once after the memory is mapped) ++ `auto id = registerPtr(start1, size);` + +register the start pointer of the segment start2 in another application with a specific id (again, this happens after +the other application maps the memory) ++ `registerPtr(id, start2, size);` + +unregister specific id (also invalidates relative pointers using this id, they cannot be used safely after this +operation anymore) ++ `unregisterPtr(id);` + +unregister all ids (also invalidates all relative pointers) ++ `unregisterAll();` + + + +### Construction and Assignment + +Assume we have an object X of type T in shared memory. Then we can construct relative pointers rp pointing to *arbitrary +registered segments*. + +default construct a logical nullptr ++ `RelativePointer rp;` + +construct relative pointer pointing to X (provided the segment containing X was registered) ++ `RelativePointer rp(&X);` + +assign relative pointer to point to X ++ `rp = &X;` + +copy construction ++ `RelativePointer rp2(rp);` + +copy assignment ++ `RelativePointer rp2 = rp;` + +### Operations + +As for relocatable pointers, operations for raw pointers also work for relative pointers. + +comparison with raw pointers ++ `if(rp != nullptr) { \\do something }` + ++ `T* raw = &X; if(p == raw) { \\do something }` + +assignment to raw (resolves the pointer in the current address space) ++ `T* raw = rp;` + +dereference ++ `T& t = *p;` + +implicitly get the raw pointer ++ `T* raw = rp;` + +explicitly get the raw pointer ++ `T* raw = rp.get();` + +access members ++ `p->func();` + +### Performance + +A relative pointer measures the distance to the pointee relatively to a registered address and therefore needs to store two +numbers, one for the distance itself and the other one to identify the segment base address to measure from. For this reason, +the operations incur slightly more overhead compared to relocatable pointers. + +## Atomic usage + +There is a technical problem using both RelocatablePointer and RelativePointer as a type in a std::atomic. This is essentially +impossible as an atomic requires its type to be copyable/movable (to be loaded) but on the other hand, this copy +constructor must be trivial, i.e. performable with a shallow memcpy. Therefore, the types used in atomic cannot implement +custom copy/move. This is not possible for RelocatablePointer and RelativePointer as both require operations performed +during copying, which cannot be done by a simple shallow copy. + +To support the use case of an atomic relocatable pointer, the template AtomicRelocatablePointer is provided. It is a +basic version of a RelocatablePointer, which lacks some of its move functionality and other operations. However, its +existing operations behave like RelocatablePointer but in an atomic way (but also incur additional overhead caused by +atomic loads and stores). Therefore, this object can be used concurrently by multiple threads, in contrast to +RelativePointer and RelocatablePointer. This atomic version also utilizes lock free atomic operations internally (when +available on the target architecture) as the data to be stored is just a 64 bit word. + +Note that this cannot be done as easily for RelativePointer as it requires more than 64 bit to store its internal +information. A construction where atomicity is guaranteed via locking could be provided in the future. + diff --git a/doc/website/design/request_response_communication.md b/doc/website/design/request_response_communication.md new file mode 100644 index 0000000000..e268a5da98 --- /dev/null +++ b/doc/website/design/request_response_communication.md @@ -0,0 +1,131 @@ +# Request/Response Communication + +## Summary + +Beside the publish-subscribe messaging pattern initially supported by iceoryx, +the request-response pattern is also widely used in the field where iceoryx is headed. +Since we aim for a good integration with ROS 2, this is also necessary to support its +`Service` communication method which is often used to control robots. + +While publish-subscribe is a data distribution pattern which continuously delivers +updates on its data, request-response is a remote procedure call and task distribution pattern +which delivers data only on demand. This means there is one or several clients +requesting a server to perform a task. + +In iceoryx, the client and server will be built on top of the same building blocks +we already use for publish-subscribe, like the `ChunkSender` and `ChunkReceiver`. + +## Terminology + +| Name | Description | +| :---------------- | :------------------------------------------------------- | +| Request-Response | communication pattern which delivers data on demand | +| Publish-Subscribe | communication pattern which continuously delivers updates on its data | +| RPC | a remote procedure call to offload task to a server | +| Client | an entity invoking RPCs | +| Server | an entity processing RPCs | + +## Design + +### Considerations + +Client and server shall be able to be used in combination with the `WaitSet` and `Listener` +to provide a means of event based communication instead of polling. + +Like with publish-subscribe, the `ServiceDescription` will be used to connect clients with a corresponding server. + +In order to support asynchronous requests, a sequence ID should be part of each request which will be copied to the corresponding response. + +### Solution + +This is an overview of the untyped `Client` and `Server` classes. + +![simple class diagram](overview_class.svg) + +The `Client` and `Server` are reusing the `ChunkSender` and `ChunkReceiver` building blocks. The `Client` uses a `ChunkSender` to send requests and a `ChunkReceiver` to get the responses while the `Server` uses a `ChunkReceiver` to get the requests and a `ChunkSender` to send the responses. + +#### Typed API + +![typed API](typed_api.svg) + +Since the `Response` is tied to a specific `Request`, the `loan` method takes a `Request` to populate the `Response` with the correct settings. +Alternatively, a `Request` could have a `createResponse` method which returns a `Response` with the correct settings. + +#### Untyped API + +![untyped API](untyped_api.svg) + +Similar to the the typed API, `loan` takes a pointer to a `RequestHeader` to populate the `ResponseHeader` with the correct settings. + +#### Client Port + +![client port](client_port.svg) + +The `ClientPortData` is located in the shared memory and contain only the data but no methods to access them. +`ClientPortUser` is the class providing the methods for the user access and `ClientPortRouDi` provides the +interface RouDi needs to connect the client to the server and to cleanup the port resources. + +#### Server Port + +![server port](server_port.svg) + +Similar to the Client Port, the Server Port has `ServerPortData` which is located in the shared memory and contain only the data but no methods to access them. +`ServerPortUser` is the class providing the methods for the user access and `ServerPortRouDi` provides the +interface RouDi needs to connect the client to the server once the server offers its service and to cleanup the port resources. + +It must be ensured that only one server with a given `ServiceDescription` can run at a time. + +#### Request/Response Header + +![rpc header](request_response_header.svg) + +Since request and response need to encode different meta-information, we also need different header for the messages. +The common data is aggregated in `RpcBaseHeader` which contains a `cxx::UniqueId` to the `ClientChunkQueueData_t` and a sequence ID. +The `cxx::UniqueId` is used to identify the queue which receives the response and used as identifier for `ChunkDistributor::deliverToQueue`. +This method will iterate over all stored queues and matches the `ChunkQueueData::m_uniqueIdOfOwner`. +As optimization, the `lastKnownQueueIndex` will be used to do a fast lookup and downgrades to a full lookup if the queue IDs don't match. +```cpp +class ChunkDistributor { + ... + cxx::expected deliverToQueue(cxx::UniqueId uniqueQueueId, uint32_t lastKnownQueueIndex, mepoo::SharedChunk chunk); + ... +}; +``` + +The sequence ID is used to match a response to a specific request if multiple requests are invoke asynchronously. +Depending on the client and server options, a request might be dropped or a server could have a worker pool +which results in sending the responses in a different order than the request were received. +The sequence ID must be set by the user and also checked by the user on response. + +The `RequestHeader` has also the option to specify a message as fire and forget, which means it won't get a response to this request. + +#### Client/Server Options + +![client and server options](client_and_server_options.svg) + +The client and server options can be used to control certain aspects of the clients and servers. +Beside setting the capacity of the queues and defining whether a client should be connected and a server offering on creation, +the behaviour for a slow client and server can be defined. +A client can ask a server to block if its response queue is full. +The server will obey if the corresponding `clientTooSlowPolicy` is set to `ConsumerTooSlowPolicy::WAIT_FOR_CONSUMER`. +If the options don't match, the client will not be connected to the server, similar to the behavior of publisher and subscriber. + +#### Client Service Discovery + +The client is guided by the following state machine. + +![client state machine](client_state_machine.svg) + +Similar to the subscriber state machine, the client passes it's response queue with the `CONNECT` CaPro message to the server. +The server will pass its request queue with the `ACK` CaPro message to the client. + +Following is a sequence diagram which shows all this cases + +![client and server service discovery](client_and_server_service_discovery.svg) + +### Code example + +## Open issues + +- integration into a gateway, e.g. the DDS gateway +- check if a Listener can be used for a timed wait for a response on the client side diff --git a/doc/design/diagrams/request_response/client_and_server_options.svg b/doc/website/images/client_and_server_options.svg similarity index 100% rename from doc/design/diagrams/request_response/client_and_server_options.svg rename to doc/website/images/client_and_server_options.svg diff --git a/doc/design/diagrams/request_response/client_and_server_service_discovery.svg b/doc/website/images/client_and_server_service_discovery.svg similarity index 100% rename from doc/design/diagrams/request_response/client_and_server_service_discovery.svg rename to doc/website/images/client_and_server_service_discovery.svg diff --git a/doc/design/diagrams/request_response/client_port.svg b/doc/website/images/client_port.svg similarity index 100% rename from doc/design/diagrams/request_response/client_port.svg rename to doc/website/images/client_port.svg diff --git a/doc/design/diagrams/request_response/client_state_machine.svg b/doc/website/images/client_state_machine.svg similarity index 100% rename from doc/design/diagrams/request_response/client_state_machine.svg rename to doc/website/images/client_state_machine.svg diff --git a/doc/website/images/favicon.ico b/doc/website/images/favicon.ico new file mode 100644 index 0000000000000000000000000000000000000000..2e89a3c7b6aa79c6b15da1528a129594ac92b6b2 GIT binary patch literal 15406 zcmeI22XGWs7{@n+UP2S3L^$aPNGO8f2q;P$!pLAj>4J!g1%@Vqg2ISIiVa3Zx(wjR z0D^#^v;jdWBO;(E2u={CNs&(C?|A`m)u@1aq`Xl^1b)H@_oDe_U(J`Vi++- ztWlzbfp!@K@)|}H!!TT~oB3KL4Pz#@gal7{4a3-4#xQD8hbFYRX-C;gYHF&>wlIVO z+E;~#L#S$7h4ww+1*l?MDM$fr=fIcH)~+gLF#*crQc_ZuQfD!`2s#d)gtVAQ-(lBY zcFM=lb{~2&`UE-^y&5$NibH+SBeWFCdVt#Og40TSWDR-?Jc|8QtHzYc$9hnE$ws9m zq%WZQqlcm&uxVd_^6{{nIyYbw=~L)Mf^?EqJ1ZM?%h0AZdJ0@8tr#|=s$12!v(csk z98`N`6O6R0YiFm8i$0E`cc4E-_k((N^{wpGuT5J$lXGyAd}phAvQd^_^`R4W2g5Yd zdVa6NDDtK4+U7L7<{1~X0?kjsgzIO>ne z;JYR|@mn_I$#sOg74z*_8#e)3y^xUL%(C==x?G(#^4bgVy)A6|^zS zyft+OQI^Ns7HXPy_tO4fc!RX$J*Y>18Kk%^{2{4ty2k5tbW!jA$!q;|pSLa4H0|z$ zx%8{CAU~%eNojwglHGMbFmA?bwrT2r~pNlzhN z815piZ*zX{zJsNyp9eG_Ekk?D1~S4O>tSGWih$u~Fq52mSKQEh26*~ZCK%Jn{^6YM`>>;VPk2d($B4%{Bh zuXj29R0j9Bqpgo%{{zEZZzHyy&>frBvoq0md-s=_G=0QEGtijQy7w~L6y0CFgKJUp zI~sjVh3I4$lo_9a+R$e)x^Iux*d7Cqz`Nvqo(1{q@HS-`y61s>Tb0p&H?Vvcu}>g| zAy)02wBhF&{OCFO2&VBEb~x$Ft}k7ue+=^PvAOKpIceuwH6GFg-$|f-UBJDmlQG~= zW6mw47lPyp9E8>IAw+CFh2t3cPypz4;QwA+q$r~I1D{1;0UQAHyGb%-1-`2}alh%O z0Mvy<`qdigefSytS1E|122AoeD(9P}(Q zwKt7M+@?=rj)C4hi@oUkbLch1dcL1Wh`+UYZ8=F3cX6&k>$=mVJ3xl^*xBJ)iMb8& zAA^(7OV@zb?+dqPhxmfUK)l+c_9b44&Q7&HYl+qxG*f4G!F)!doS0`2ulB9Ur0aq9 zm5a!ikAy#``y@`SkGG+<$Eb?Ff|jg>I!^m@x<6vnoT_)>xZ*|YotEglX*v{f(kJ~E zgjk3B)c#LrV>&z58IksZmti=`E`fYe9;0$}NHI~bI#deOXJ*QXRqKmd=>2HLuenL* zfP295H$tjar))4<=e1QlxUa`92I`lGfp+EL*y*D9LKOeLw*EkyN-7opF*mX{% zbF-`H!{`L`FmxyHy8da(3sP?#x`2OKxOw_4LEjre=Y9IUR(oh&%RbQaDF@w1{|r87 zPh?eBzj5n~ehS2sw|o}ZzfwQp#_Qi6pT%G$tbzex#`q7($6yIu!?%@a-rWFgKxYU# zSLg!i)`{3=z#7QZZ#@2Sn|b;xOnmdvozaKUrf3Z5p6Z-Gkf@&CpN8l!;2g{XpYI|0 zSKu$|)ir%Pq3KU!^I6dOo4@G2cn)-*0}Abbq(UNWC#`3=1)76<%%gi^zJ%`QMBHW{ z^rzoGYS8yaw2yfFY3v{HDX`AyIjp2e&vC1&pf#V3gYsl`Eqd=~p^KXS!_oAU4|;(9 zCq~aUonT)9IyVa{)Voe4J+sE7cY#Fjj%@dU{Ctb95@y_a=|ksvJJHM0lhBr9#N%Jz zYWM#VQ7;tH*wJ&-cSP$%eHYKeFz7{DSpWY(ADZ*Gp>@A>PkulL60P@^f;l$@s;hCM zYkdM5f`0dS5u45*^T;=>y-ITU)~f(EO*dqjzy6XpB5h zjJp43%$U+d--3y-2TPXzKa1715`*?14M6{U*@(FILa=v!8g(=#E&tcj%1>6pq=GWoWPq(IaI;M>FM_d&_d!KaIfGL@nA6FT&gmhH@Z`1DDh)e7K-Dn@drv03L z!_?a99dx!m<0HPN#OJeyBCluC2j;<4vM+n`9nqT0DnYjSeTZ1JhtM7-O~`9cq;G*l z=Y5wz`?C{pj57Vdna@7W?m1_Lop{>8A#e+^X$^7Kll4)QNwlW5Jg3Quc>R44pw9I= z64OynusT!K?>g=)w*8W|EKitKhk+((O$G1=)KkW&|X+^x`njf2|XXJL*gPG zkJ&%Iv|lI$rJw@&YM|fK+LG5C|1oKO&yIt}`6?I$`?1ePOEl&ekawB=Mncnne*DCv z8-dQV^c%rw^mJGTTj2;?z-}j3DVPIVn`+;+lyoAb?PnqpyZXe3_5@l>mxbF%YmHnJ nYG)9ApS9o8ddZRVhDcp=Zoj#AAomXB-htdZkb4Khy#xONQWqeZ literal 0 HcmV?d00001 diff --git a/doc/design/diagrams/iceoryx_components_diagram_nextgen_release.svg b/doc/website/images/iceoryx_components_diagram_nextgen_release.svg similarity index 100% rename from doc/design/diagrams/iceoryx_components_diagram_nextgen_release.svg rename to doc/website/images/iceoryx_components_diagram_nextgen_release.svg diff --git a/doc/design/diagrams/service-discovery/overview-alternative-b.svg b/doc/website/images/overview-alternative-b.svg similarity index 100% rename from doc/design/diagrams/service-discovery/overview-alternative-b.svg rename to doc/website/images/overview-alternative-b.svg diff --git a/doc/design/diagrams/service-discovery/overview-alternative-d.svg b/doc/website/images/overview-alternative-d.svg similarity index 100% rename from doc/design/diagrams/service-discovery/overview-alternative-d.svg rename to doc/website/images/overview-alternative-d.svg diff --git a/doc/design/diagrams/request_response/overview_class.svg b/doc/website/images/overview_class.svg similarity index 100% rename from doc/design/diagrams/request_response/overview_class.svg rename to doc/website/images/overview_class.svg diff --git a/doc/design/diagrams/puml_github_workflow.svg b/doc/website/images/puml_github_workflow.svg similarity index 100% rename from doc/design/diagrams/puml_github_workflow.svg rename to doc/website/images/puml_github_workflow.svg diff --git a/doc/design/diagrams/request_response/request_response_header.svg b/doc/website/images/request_response_header.svg similarity index 100% rename from doc/design/diagrams/request_response/request_response_header.svg rename to doc/website/images/request_response_header.svg diff --git a/doc/design/diagrams/service-discovery/sequence-diagram-alternative-b.svg b/doc/website/images/sequence-diagram-alternative-b.svg similarity index 100% rename from doc/design/diagrams/service-discovery/sequence-diagram-alternative-b.svg rename to doc/website/images/sequence-diagram-alternative-b.svg diff --git a/doc/design/diagrams/service-discovery/sequence-diagram-alternative-d.svg b/doc/website/images/sequence-diagram-alternative-d.svg similarity index 100% rename from doc/design/diagrams/service-discovery/sequence-diagram-alternative-d.svg rename to doc/website/images/sequence-diagram-alternative-d.svg diff --git a/doc/design/diagrams/request_response/server_port.svg b/doc/website/images/server_port.svg similarity index 100% rename from doc/design/diagrams/request_response/server_port.svg rename to doc/website/images/server_port.svg diff --git a/doc/design/diagrams/request_response/typed_api.svg b/doc/website/images/typed_api.svg similarity index 100% rename from doc/design/diagrams/request_response/typed_api.svg rename to doc/website/images/typed_api.svg diff --git a/doc/design/diagrams/request_response/untyped_api.svg b/doc/website/images/untyped_api.svg similarity index 100% rename from doc/design/diagrams/request_response/untyped_api.svg rename to doc/website/images/untyped_api.svg diff --git a/mkdocs.yml b/mkdocs.yml index 0d560f5abe..fb5658bab9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -17,14 +17,18 @@ markdown_extensions: - pymdownx.emoji: emoji_index: !!python/name:materialx.emoji.twemoji emoji_generator: !!python/name:materialx.emoji.to_svg + - pymdownx.tasklist: + custom_checkbox: true plugins: - awesome-pages - search + - autolinks theme: name: 'material' custom_dir: doc/website/overrides font: false logo: images/iceoryx-logo-white.png + favicon: images/favicon.ico features: - navigation.indexes extra: @@ -32,4 +36,4 @@ extra: provider: mike extra_css: - stylesheets/extra.css - \ No newline at end of file +use_directory_urls: true diff --git a/tools/website/export-docu-to-website.sh b/tools/website/export-docu-to-website.sh index 50c5d3fc87..566b0ea079 100755 --- a/tools/website/export-docu-to-website.sh +++ b/tools/website/export-docu-to-website.sh @@ -18,7 +18,8 @@ # Necessary tools: # mkdocs-awesome-pages-plugin, v2.6.0 -# mkdocs-material, v8.1.3-insiders-4.5.0q +# mkdocs-autolinks-plugin, v0.4.0 (with svg patch) +# mkdocs-material, v8.1.3-insiders-4.5.0 # markdown-include, v0.6.0 # mkdocs, v1.2.3 # Doxygen, v1.9.1 From b9408f1b25e4538142be6a3b8ac668f79a62f885 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Thu, 17 Feb 2022 15:04:19 +0100 Subject: [PATCH 09/12] iox-#743 Rename 'tutorials' back to 'examples' and adapt links to work with 'autolinks' plugin Signed-off-by: Simon Hoinkis --- doc/website/.pages | 2 +- ...nd-error-values-are-returned-in-iceoryx.md | 125 ------- doc/website/concepts/architecture.md | 2 +- doc/website/design/chunk_header.md | 287 ---------------- doc/website/design/error-handling.md | 318 ------------------ doc/website/design/listener.md | 317 ----------------- doc/website/design/lockfree_queue.md | 135 -------- doc/website/design/relocatale_pointer.md | 215 ------------ .../design/request_response_communication.md | 131 -------- doc/website/design/service-discovery.md | 5 - doc/website/{tutorials => examples}/.pages | 1 + .../{tutorials => examples}/callbacks.md | 0 .../{tutorials => examples}/callbacks_in_c.md | 0 .../{tutorials => examples}/complexdata.md | 3 +- .../ice_access_control.md | 0 .../{tutorials => examples}/icecrystal.md | 0 .../{tutorials => examples}/icedelivery.md | 0 .../icedelivery_in_c.md | 0 .../{tutorials => examples}/icedocker.md | 0 .../{tutorials => examples}/iceensemble.md | 0 .../{tutorials => examples}/icehello.md | 0 .../{tutorials => examples}/iceoptions.md | 0 .../{tutorials => examples}/iceperf.md | 0 doc/website/{tutorials => examples}/index.md | 2 +- .../{tutorials => examples}/singleprocess.md | 0 .../{tutorials => examples}/user_header.md | 2 +- .../{tutorials => examples}/waitset.md | 0 .../{tutorials => examples}/waitset_in_c.md | 0 doc/website/getting-started/overview.md | 16 +- .../getting-started/what-is-iceoryx.md | 17 +- doc/website/images/favicon.ico | Bin 15406 -> 15406 bytes 31 files changed, 23 insertions(+), 1555 deletions(-) delete mode 100644 doc/website/advanced/how-optional-and-error-values-are-returned-in-iceoryx.md delete mode 100644 doc/website/design/chunk_header.md delete mode 100644 doc/website/design/error-handling.md delete mode 100644 doc/website/design/listener.md delete mode 100755 doc/website/design/lockfree_queue.md delete mode 100644 doc/website/design/relocatale_pointer.md delete mode 100644 doc/website/design/request_response_communication.md delete mode 100644 doc/website/design/service-discovery.md rename doc/website/{tutorials => examples}/.pages (94%) rename doc/website/{tutorials => examples}/callbacks.md (100%) rename doc/website/{tutorials => examples}/callbacks_in_c.md (100%) rename doc/website/{tutorials => examples}/complexdata.md (58%) rename doc/website/{tutorials => examples}/ice_access_control.md (100%) rename doc/website/{tutorials => examples}/icecrystal.md (100%) rename doc/website/{tutorials => examples}/icedelivery.md (100%) rename doc/website/{tutorials => examples}/icedelivery_in_c.md (100%) rename doc/website/{tutorials => examples}/icedocker.md (100%) rename doc/website/{tutorials => examples}/iceensemble.md (100%) rename doc/website/{tutorials => examples}/icehello.md (100%) rename doc/website/{tutorials => examples}/iceoptions.md (100%) rename doc/website/{tutorials => examples}/iceperf.md (100%) rename doc/website/{tutorials => examples}/index.md (68%) rename doc/website/{tutorials => examples}/singleprocess.md (100%) rename doc/website/{tutorials => examples}/user_header.md (60%) rename doc/website/{tutorials => examples}/waitset.md (100%) rename doc/website/{tutorials => examples}/waitset_in_c.md (100%) diff --git a/doc/website/.pages b/doc/website/.pages index 33b758f7fe..a1f9abefb5 100644 --- a/doc/website/.pages +++ b/doc/website/.pages @@ -2,7 +2,7 @@ nav: - index.md - getting-started - concepts - - tutorials + - examples - design - FAQ.md - advanced diff --git a/doc/website/advanced/how-optional-and-error-values-are-returned-in-iceoryx.md b/doc/website/advanced/how-optional-and-error-values-are-returned-in-iceoryx.md deleted file mode 100644 index 8396fb0c9b..0000000000 --- a/doc/website/advanced/how-optional-and-error-values-are-returned-in-iceoryx.md +++ /dev/null @@ -1,125 +0,0 @@ -# How optional and error values are returned in iceoryx - -Many parts of the iceoryx C++ API follow a functional programming approach and allow the user to specify functions -which handle the possible cases, e.g. what should happen when data is received. - -This is very flexible but requires using the monadic types `cxx::expected` and `cxx::optional`, which we -introduce in the following sections. - -## Optional - -The type `iox::cxx::optional` is used to indicate that there may or may not be a value of a specific type `T` -available. This is essentially the 'maybe [monad](https://en.wikipedia.org/wiki/Monad_(functional_programming))' in -functional programming. Assuming we have some optional (usually the result of some computation) - -```cpp -optional result = someComputation(); -``` - -we can check for its value using - -```cpp -if(result.has_value()) -{ - auto value = result.value(); - // do something with the value -} -else -{ - // handle the case that there is no value -} -``` - -A shorthand to get the value is - -```cpp -auto value = *result; -``` - -!!! attention - Accessing the value if there is no value terminates the application, so it must be checked beforehand. - -We can achieve the same with the functional approach by providing a function for both cases. - -```cpp -result.and_then([](int& value) { /*do something with the value*/ }) - .or_else([]() { /*handle the case that there is no value*/ }); -``` - -Notice that we get the value by reference, so if a copy is desired it has to be created explicitly in the -[lambda](https://en.wikipedia.org/wiki/Anonymous_function#C++_(since_C++11)) or function we pass. - -The optional can be initialized from a value directly - -```cpp -optional result = 73; -result = 37; -``` - -If the optional is default initialized, it is automatically set to its null value of type `iox::cxx::nullopt_t`. -This can be also done directly by using the constant `iox::cxx::nullopt` - -```cpp -result = iox::cxx::nullopt; -``` - -For a complete list of available functions see -[`optional.hpp`](https://github.com/eclipse-iceoryx/iceoryx/blob/master/iceoryx_hoofs/include/iceoryx_hoofs/cxx/optional.hpp). -The `iox::cxx::optional` behaves like the [`std::optional`](https://en.cppreference.com/w/cpp/utility/optional) -except that it does not throw exceptions and has no undefined behavior. - -## Expected - -`iox::cxx::expected` generalizes `iox::cxx::optional` by admitting a value of another type `E` instead of -no value at all, i.e. it contains either a value of type `T` or `E`. In this way, `expected` is a special case of -the 'either monad'. It is usually used to pass a value of type `T` or an error that may have occurred, i.e. `E` is the -error type. - -For more information on how it is used for error handling see -[error-handling.md](https://github.com/eclipse-iceoryx/iceoryx/blob/master/doc/design/error-handling.md). - -Assume we have `E` as an error type, then we can create a value - -```cpp -iox::cxx::expected result(iox::cxx::success(73)); -``` - -and use the value or handle a potential error - -```cpp -if (!result.has_error()) -{ - auto value = result.value(); - // do something with the value -} -else -{ - auto error = result.get_error(); - // handle the error -} -``` - -If we need an error value, we set - -```cpp -result = iox::cxx::error(errorCode); -``` - -which assumes that `E` can be constructed from an `errorCode`. - -We again can employ a functional approach like this: - -```cpp -auto handleValue = [](int& value) { /*do something with the value*/ }; -auto handleError = [](E& value) { /*handle the error*/ }; -result.and_then(handleValue).or_else(handleError); -``` - -There are more convenience functions such as `value_or` which provides the value or an alternative specified by the -user. These can be found in -[`expected.hpp`](https://github.com/eclipse-iceoryx/iceoryx/blob/master/iceoryx_hoofs/include/iceoryx_hoofs/cxx/expected.hpp). - -Note that when we move an `expected`, the origin contains a moved `T` or `E`, depending on the content before the move. -This mirrors the behavior of moving the content out of the `iox::cxx::expected` like with -`auto foo = std::move(bar.value());` with `bar` being an `iox::cxx::expected`. -Like all objects, `T` and `E` must therefore be in a well defined state after the move. diff --git a/doc/website/concepts/architecture.md b/doc/website/concepts/architecture.md index c7d8fc7341..c79bec1ac6 100644 --- a/doc/website/concepts/architecture.md +++ b/doc/website/concepts/architecture.md @@ -1 +1 @@ -# Eclipse iceoryx architecture +# Architecture diff --git a/doc/website/design/chunk_header.md b/doc/website/design/chunk_header.md deleted file mode 100644 index c792cf41ec..0000000000 --- a/doc/website/design/chunk_header.md +++ /dev/null @@ -1,287 +0,0 @@ -# Chunk-Header - -## Summary - -Chunks are the transport capsules in iceoryx. They store data from a publisher as payload and are sent to one or more subscriber. Furthermore, there is some meta information which is stored alongside the payload, e.g. the size and the origin of the chunk. This data is composed in the `ChunkHeader` and located at the front of the chunk. Custom meta information can be added to extend the data from the `ChunkHeader` and tailor the chunk to specific use cases. While this makes the chunk layout more complex, this complexity would otherwise be distributed at different locations all over the code base, e.g. in the request/response feature. Additionally, the adjustments for the user-header makes arbitrary alignments of the user-payload trivial to implement. - -## Terminology - -| Name | Description | -| :---------------- | :------------------------------------------------------- | -| Chunk | a piece of memory | -| Chunk-Header | contains meta information related to the chunk | -| Chunk-Payload | the part of the chunk for the payload which is split into user-header and user-payload | -| User-Header | contains custom meta information, e.g. timestamps | -| User-Payload | the user data with custom alignment | -| Back-Offset | offset stored in front of the user-payload to calculate back to the chunk-header (for the most simple case it will overlap with the user-payload offset stored in the Chunk-Header) | - -Framing with terminology -``` -+=============================================================================+ -| Chunk | -+===================+=========================================================+ -| | Chunk-Payload | -| Chunk-Header +===============+=======+====================+============+ -| | User-Header | ¦ ᶺ | User-Payload | Padding | -+===================+===============+=====|=+====================+============+ - └ Back-Offset -``` - -## Design - -### Considerations - -- it's not uncommon to record chunks for a later replay -> detect incompatibilities on replay -- iceoryx runs on multiple platforms -> endianness of recorded chunks might differ -- for tracing, a chunk should be uniquely identifiable -> store origin and sequence number -- the chunk is located in the shared memory, which will be mapped to arbitrary positions in the address space of various processes -> no absolute pointer are allowed -- in order to reduce complexity, the alignment of the user-header must not exceed the alignment of the `ChunkHeader` - -### Solution - -#### ChunkHeader Definition -``` -class ChunkHeader -{ - uint32_t chunkSize; - uint8_t chunkHeaderVersion; - uint8_t reserved{0}; - uint16_t userHeaderId; - uint64_t originId; - uint64_t sequenceNumber; - uint32_t userHeaderSize{0U}; - uint32_t userPayloadSize{0U}; - uint32_t userPayloadAlignment{1U}; - uint32_t userPayloadOffset; -}; -``` - -- **chunkSize** is the size of the whole chunk -- **chunkHeaderVersion** is used to detect incompatibilities for record&replay functionality -- **reserved** is currently not used and set to `0` -- **userHeaderId** is currently not used and set to `NO_USER_HEADER` -- **originId** is the unique identifier of the publisher the chunk was sent from -- **sequenceNumber** is a serial number for the sent chunks -- **userPayloadSize** is the size of the chunk occupied by the user-header -- **userPayloadSize** is the size of the chunk occupied by the user-payload -- **userPayloadAlignment** is the alignment of the chunk occupied by the user-payload -- **userPayloadOffset** is the offset of the user-payload relative to the begin of the chunk - -#### Framing - -For back calculation from the user-payload pointer to the `ChunkHeader` pointer, the user-payload offset must be accessible from a defined position relative to the user-payload. Lets call this `back-offset`. This is solved by storing the offset in the 4 bytes in front of the user-payload. In case of a simple layout where the `ChunkHeader` is adjacent to the user-payload, this nicely overlaps with the position of `userPayloadOffset` and no memory is wasted. In more complex cases, the offset has to be stored a second time. If the user-payload alignment requires some padding from the header extension, this memory is used to store the offset. - -1. No user-header and user-payload alignment doesn't exceed the `ChunkHeader` alignment - -``` - sizeof(ChunkHeader) userPayloadSize -|------------------>|--------------------->| -| | | -+===================+======================+==================================+ -| Chunk-Header ¦ * | User-Payload | Padding | -+===================+======================+==================================+ -| | | -| userPayloadOffset | | -|------------------>| | -| chunkSize | -|---------------------------------------------------------------------------->| - -*) userPayloadOffset from ChunkHeader and back-offset are overlapping -``` - -2. No user-header and user-payload alignment exceeds the `ChunkHeader` alignment - -``` - sizeof(ChunkHeader) back-offset userPayloadSize -|------------------>| |<---|------------------->| -| | | | | -+===================+=======================+====================+============+ -| Chunk-Header | ¦ | User-Payload | Padding | -+===================+=======================+====================+============+ -| | | -| userPayloadOffset | | -|------------------------------------------>| | -| | -| chunkSize | -|---------------------------------------------------------------------------->| -``` - -Depending on the address of the chunk there is the chance that `ChunkHeader` is the still adjacent to the user-payload. In this case, the framing looks exactly like in case 1. - -3. User-Header is used - -``` - sizeof(ChunkHeader) back-offset userPayloadSize -|------------------>| |<---|------------------->| -| | | | | -+===================+===============+=======+====================+============+ -| Chunk-Header | User-Header | ¦ | User-Payload | Padding | -+===================+===============+=======+====================+============+ -| | | -| userPayloadOffset | | -|------------------------------------------>| | -| | -| chunkSize | -|---------------------------------------------------------------------------->| -``` - -#### User-Payload Offset Calculation - -1. No user-header and user-payload alignment doesn't exceed the `ChunkHeader` alignment - -``` -userPayloadOffset = sizeof(ChunkHeader); -``` - -2. No user-header and user-payload alignment exceeds the `ChunkHeader` alignment, which means the user-payload is either adjacent to the `ChunkHeader` or there is a padding with at least the size of `ChunkHeader` alignment in front of the user-payload and therefore enough space to store the `back-offset` - -``` -chunkHeaderEndAddress = addressof(chunkHeader) + sizeof(chunkHeader); -alignedUserPayloadAddress = align(chunkHeaderEndAddress, userPayloadAlignment); -userPayloadOffset = alignedUserPayloadAddress - addressof(chunkHeader); -``` - -3. User-Header is used - -``` -chunkHeaderEndAddress = addressof(chunkHeader) + sizeof(chunkHeader) + sizeof(userHeader); -anticipatedBackOffsetAddress = align(chunkHeaderEndAddress, alignof(userPayloadOffset)); -unalignedUserPayloadAddress = anticipatedBackOffsetAddress + sizeof(userPayloadOffset); -alignedUserPayloadAddress = align(unalignedUserPayloadAddress, userPayloadAlignment); -userPayloadOffset = alignedUserPayloadAddress - addressof(chunkHeader); -``` - -#### Required Chunk Size Calculation - -In order to fit the user-header and the user-payload into the chunk, a worst case calculation has to be done. We can assume that a chunk is sufficiently aligned for the `ChunkHeader`. - -1. No user-header and user-payload alignment doesn't exceed the `ChunkHeader` alignment - -``` -chunkSize = sizeof(chunkHeader) + userPayloadSize; -``` - -2. No user-header and user-payload alignment exceeds the `ChunkHeader` alignment - -Worst case scenario is when a part of the `ChunkHeader` crosses the user-payload alignment boundary, so that the user-payload must be aligned to the next boundary. The following drawing demonstrates this scenario. - -``` - ┌ back-offset - +===============+==========|+===============================+ - | ChunkHeader | ¦ᵛ| User-Payload | - +===============+===========+===============================+ - -⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- ChunkHeader alignment boundaries -⊥ ⊥ ⊥ ⊥ <- user-payload alignment boundaries - <-----------|---------------> - pre user-payload |-------------------------------> - user-payload alignment user-payload size - alignment - overhang -``` - -The following formula is used to calculate the required chunk size. - -``` -preUserPayloadAlignmentOverhang = sizeof(chunkHeader) - alignof(chunkHeader); -chunkSize = preUserPayloadAlignmentOverhang + userPayloadAlignment + userPayloadSize; -``` - -3. User-Header is used - -Similar to case 2, but in this case it is the `back-offset` which might cross the user-payload alignment boundary. - -``` - ┌ back-offset with same alignment - | as userPayloadOffset - +===============+===========+==============|+===============================+ - | ChunkHeader | UserHeader| ¦ᵛ| User-Payload | - +===============+===========+===============+===============================+ - - ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- userPayloadOffset alignment boundaries -⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- ChunkHeader alignment boundaries -⊥ ⊥ ⊥ ⊥ <- userPayload alignment boundaries - <---------------------------|---------------> - pre user-payload alignment user-payload |-------------------------------> - overhang alignment user-payload size -``` - -The following formula is used to calculate the required chunk size. - -``` -chunkHeaderSize = sizeof(chunkHeader) + sizeof(userHeader) -preUserPayloadAlignmentOverhang = align(chunkHeaderSize, alignof(userPayloadOffset)); -maxPadding = max(sizeof(userPayloadOffset), userPayloadAlignment); -chunkSize = preUserPayloadAlignmentOverhang + maxPadding + userPayloadSize; -``` - -#### Accessing Chunk-Header Extension - -The `ChunkHeader` has a template method to get user-header. There is a risk to use this wrong by accident, but there is currently no better solution for this. - -Since the user-header is always adjacent to the `ChunkHeader`, the formula to get the user-header is: -``` -userHeader = addressOf(chunkHeader) + sizeof(chunkHeader); -``` - -#### ChunkHeader Methods - -- `void* userPayload()` returns a pointer to the user-payload -- `template T* userHeader()` returns a pointer to the user-header -- `static ChunkHeader* fromUserPayload(const void* const userPayload)` returns a pointer to the `ChunkHeader` associated to the user-payload - -#### Integration Into Publisher/Subscriber API - -The `Publisher` has additional template parameters for the user-header and user-payload alignment. Alternatively this could also be done with the `allocate` method, but that increases the risk of using it wrong by accident. - -Additionally, to fully integrate the user-header into the regular control flow, a pointer to a `ChunkHeaderHook` can be passed to the `Publisher` c'tor. - -The `ChunkHeaderHook` is not in the shared memory and has the following virtual methods: -- `virtual void allocateHook(ChunkHeader& chunkHeader)` can be used to initialize the user-header -- `virtual void deliveryHook(ChunkHeader& chunkHeader)` can be used to set members of the user-header, e.g. a timestamp - -Furthermore, the `Publisher` and `Subscriber` have access to the `ChunkHeader` and can use the `userHeader()` method to gain access to the user-header. - -#### Pitfalls & Testing - -- when the user-payload is adjacent to the `ChunkHeader`, it must be ensured that `userPayloadOffset` overlaps with the `back-offset`, which is `sizeof(userPayloadOffset)` in front of the user-payload -- to simplify calculation, it is assumed that the alignment of the user-header doesn't exceed the alignment of the `ChunkHeader`. This has to be enforced with an `assert` - -## Open Issues - -- a `ChunkHeaderHook` could be used on the publisher side -``` -template -class MyChunkHeaderHook : public ChunkHeaderHook -{ - public: - void deliveryHook(ChunkHeader& chunkHeader) override - { - chunkHeader.userHeader().timestamp = myTimeProvider::now(); - } -}; - -auto userHeaderHook = MyChunkHeaderHook(); -auto pub = iox::popo::Publisher(serviceDescription, userHeaderHook); -``` - - alternatively, instead of the ChunkHeaderHook class, the publisher could have a method `registerDeliveryHook(std::function)` - - allow the user only read access to the `ChunkHeader` and write access to the `UserHeader` -- user defined sequence number - - this can probably be done by a `ChunkHeaderHook` - - alternatively, a flag could be introduce into the `ChunkHeader` -- user defined timestamp - - this should probably be in a user-header -- mempool configuration - - currently we specify the chunk-payload size and the size of the `ChunkHeader` is added automatically - - with the new approach, part of the specified chunk-payload size might be used for the user-header - - part of the specified chunk-payload might also be used as padding for the user-payload alignment - - the user will continue to specify the chunk-payload; if a user-header or custom user-payload alignment is used, the user needs to take this into account -- is it necessary to store a flag in `ChunkHeader` if a user extension is used? - - we could maintain a list of known user-header IDs or ranges of IDs similar to `IANA` https://tools.ietf.org/id/draft-cotton-tsvwg-iana-ports-00.html#privateports - - the IDs could be stored in the `ChunkHeader` and everything with an ID larger than `0xC000` is free to use - - to make this somewhat save, the `ChunkHeaderHook` must be mandatory with e.g. a `virtual uint16_t getId() = 0;` method which will be called in the `ChunkSender` - - alternatively, the user-header struct must have a `constexpr uint16_t USER_HEADER_ID`; if it's not present, we could set the ID to `0xC000` which is the first ID free to use -- do we want to store the version of the user extension in the `ChunkHeader`, similarly to `chunkHeaderVersion` -- for record&replay the user-header is totally opaque, which might lead to problems if e.g. a timestamp is stored in the user-header and needs to be updated for the replay - - if we maintain a list of known user-header IDs and also store the user-header version, a record&replay framework could implement the required conversions -- for record&replay it is necessary to store the alignment of the user-payload; decide if a uint16_t should be used of if just the alignment power should be stored in a uint8_t diff --git a/doc/website/design/error-handling.md b/doc/website/design/error-handling.md deleted file mode 100644 index 9f195e782d..0000000000 --- a/doc/website/design/error-handling.md +++ /dev/null @@ -1,318 +0,0 @@ -# Logging and error handling - -## Logging - -Logging is performed by the Iceoryx internal logger. The logger API implements a subset of the Autosar log and trace interface ``ara::log``. -The logger is used internally to record errors and general system runtime information. It can also be used by the user in application code for the same purpose. - -### Logger - -The logger is responsible for logging information about the state of the system to a configurable stream (includes files). - -The logger is thread-safe and can hence be safely used from multiple threads concurrently. Currently the logger is synchronous. - -### Log levels - -The following log levels are supported, ordered by the amount of information displayed from highest to lowest. - -* ``VERBOSE`` - all available information is displayed -* ``DEBUG`` - information to support debugging on developer side -* ``INFO`` - run state information for the user -* ``WARN`` - indicates a potential problem which requires investigation -* ``ERR`` - an error occurred that may be handled on application side or by RouDi -* ``FATAL`` - an error occurred and RouDi is unable to continue -* ``OFF`` - no logging information - -For ``ERR`` and ``FATAL`` see also error levels ``MODERATE``, ``SEVERE`` (logged with ``LogErr``) and ``FATAL`` (logged with ``LogFatal``) in [Error Levels](#Error-Levels). The levels ``ERR`` and ``FATAL`` are only supposed to be used together with the error handler, i.e. need to be accompanied with a corresponding error handler call (currently this cannot be enforced). - -## Error handling - -Errors are considered to be system states that should not be reached regularly and usually are the result of an external failure, such as when the OS is unable to provide a certain resource (e.g. a semaphore) or an application does not respond. In contrast, regular behavior such as a receiver receiving no data when none was sent is not an error. On the other hand, losing data that was sent would be considered an error. - -There are two general approaches to deal with errors: -1. using exceptions - -2. return codes combined with control flow statements and a central instance to handle errors that cannot be mitigated otherwise (the error handler). - -In Iceoryx we use the latter approach. - -### Exceptions -The use of exceptions in Iceoryx is prohibited, including third party code that may throw them. This is due to the following reasons: -* In many implementations exception handling requires dynamic memory (even when the exceptions themselves are not generated dynamically via e.g. new). -* Exception handling is not deterministic with respect to runtime. -* Exception handling may cause a (slight) runtime overhead even when no exceptions are thrown. -* In general it is not possible to incorporate (complete) information about all exceptions that can be thrown in the signature of functions. This makes it hard for the caller to decide whether some exception needs to be handled. -* Overuse of exceptions often leads to convoluted try-catch blocks which makes the code hard to maintain. - -#### Use of noexcept -All functions are marked as ``noexcept``. Note that this does not mean that exceptions cannot be thrown inside such a function. Instead when an exception is thrown inside the function (either directly or indirectly by another function) this exception is not propagated further and ``std::terminate`` will be invoked, calling the terminate handler. The goal is that this cannot happen under any circumstances during runtime. - -To this end it is necessary to eliminate the use of all potentially throwing (third party) functions throughout the codebase. Since many STL functions may throw, these cannot be used either and their functionality needs to be reimplemented without the use of exceptions. In particular anything allocating dynamic memory may throw a ``std::bad_alloc`` exception when memory is exhausted. - -#### Alternatives to exceptions -As an alternative to exceptions we use the error handler and a variation of return codes in the form of ``cxx::expected``, described below. ``cxx::expected`` can be used to communicate the error to the caller, who has to decide whether to handle the error itself or propagate it further (e.g. as another ``cxx::expected``). Error handling itself is performed by the error handler which handles errors occurring in the subcomponents of ``iceoryx::posh``. - -### Error Handler -The error handler is called internally when an error is detected in the Iceoryx middleware daemon (RouDi) or the Iceoryx runtime. The error handler should only be called in exceptional situations (invalid access errors, out of resources etc.) and not in circumstances that occur regularly (it is sort of an exception replacement). - -If the exceptional situation can be resolved without calling the error handler (e.g. by delegating an appropriate return value to the caller), this should be preferred (since the error handler is a last resort mechanism). - -It is not supposed to be called by applications at any time. - -### Technical Requirements -* The error handler must be reentrant. -* The error handler must be thread-safe. -* The error handler uses the logger but the logger cannot depend on the error handler. -* When a fatal error is detected and termination is required, the reporting thread shuts down the RouDi gracefully. -* A custom error handler function can be installed (e.g. for testing). -* When the reaction on an error is just logging, computation in other threads shall not be influenced. -* If the error does not require termination, the error handler must return eventually. - -### Error Levels -The following error levels are supported. - -* ``MODERATE`` -* ``SEVERE`` -* ``FATAL`` - -#### MODERATE -A recoverable error. Leads to an error log entry (``LogErr``) and continues execution. - -**Example:** -1) RouDi receives an unexpected message and discards it. The remaining communication proceeds normally. -2) A port requested by an application cannot be provided due to e.g. resource exhaustion. - -#### SEVERE -RouDi may continue but applications may be compromised or the functionality reduced. Leads to an error log entry (``LogErr``) and assert, terminating execution in debug mode. The handler must return to be able to continue execution in release mode. - -**Example:** -A message queue is overflowing and messages are lost. RouDi can continue but lost data may affect applications. - -#### FATAL -RouDi cannot continue and will shut down. Leads to an error log entry (``LogFatal``), assert and calls ``std::terminate``, terminating execution in debug and release mode. -Before calling terminate, a callback is invoked (if configured), which can execute specific error handling code (e.g. call a 3rd party error handler). -The handler is not required to return here (since this may not be always possible or reasonable). The reporting code should still try to proceed to a safe state if possible in order to improve testability in case of such errors. - -A fatal error in the runtime terminates the application. - -**Example:** -RouDi is unable to allocate sufficient shared memory. - -### Error Codes and Additional Information - -Currently error codes are used to identify the location of an error. These are provided as an enum in error_handling.hpp. To uniquely identify an error location, these have to be different for each error. While this does not necessarily have to be the case, it is currently advised as there is no additional information (i.e. line, file) yet which allows to distinguish between different errors with the same code. - -In addition a user callback may be provided. Currently it cannot take arguments directly but since passing a generic callable with signature ``void(void)`` is possible, a lambda or functor object containing additional arguments can be provided. - -### Expects and Ensures - -These assert-like constructs are used to document assumptions in the code which are checked (at least) in debug mode. Currently they are always active, i.e. also checked in release mode. If the condition is violated they print the condition, the location of occurrence in the code and terminate the program execution. - -Since they are not necessarily active in release mode, they cannot be used to handle errors. Their purpose is to detect misuse or bugs of the API early in debug mode or to verify a result of an algorithm before returning. In this way, assumptions of the developer are made explicit without causing overhead when not needed. Therefore errors to be caught by Expects and Ensures are considered bugs and need to be fixed or the underlying assumptions and algorithms changed. This is in contrast to errors which are expected to occur during runtime which are handled by the error handler (i.e. a system resource cannot be obtained). - -Although Expects end Ensures behave the same, the former is used to signify a precondition (e.g. arguments of a function) is checked, while the latter indicates a postcondition check (e.g. result of a function before returning) - -Examples include expecting pointers that are not null (as input, intermediate or final result) or range checks of variables. - -### cxx::expected - -``cxx::expected`` is a template which either holds the result of the computation of type ``T`` or an object of error type ``E``. The latter can be used to obtain additional information about the error, e.g. an error code. -In a way this extends error codes and may act as kind of a replacement of exceptions. It is usually used as return type of functions which may fail for various reasons and should be used if the error is supposed to be delegated to the caller and handled in the caller context. -It is also possible to further propagate the error to the next function in the call-stack (since this must be done explicitly, which is comparable to rethrowing an exception). - -It is possible to use the ``nodiscard`` option to force the user to handle the returned ``cxx::expected``. -If the error cannot be handled at a higher level by the caller, the error handler needs to be used. - -Examples include wrapping third party API functions that return error codes or obtaining a value from a container when this can fail for some reason (e.g. container is empty). If no additional information about the error is available or required, ``cxx::optional`` can be used instead. - -### Error Handling in posh - -Error logging shall be done by the logger only, no calls to ``std::cerr`` or similar should be performed. - -All the methods presented (``cxx::expected``, ``Expects`` and ``Ensures`` and the error handler) can be used in posh. The appropriate way depends on the type of error scenario (cf. the respective sections for examples). The error handler should be considered the last option. - -### Error Handling in hoofs - -Error logging is currently done by calls to ``std::cerr``. In the future those might be redirected to the logger. - -The error handler cannot be used in hoofs. - -Whether it is appropriate to use ``cxx::expected`` even if STL compatibility is broken by doing so depends on the circumstances and needs to be decided on a case-by-case basis. If the function has no STL counterpart ``cxx::expected`` can be used freely to communicate potential failure to the caller. - -It should be noted that since currently `Expects` and `Ensures` are active at release mode, prolific usage of these will incur a runtime cost. Since this is likely to change in the future, it is still advised to use them to document the developer's intentions. - -Do not use `std::terminate` directly but use assert-like constructs such as ``Expects`` or ``Ensures``. - -### Interface for 3rd Party Code - -The Error handler as well as logger shall be able to use or redirect to 3rd party error handling or logging libraries in the future. Currently this is neither fully supported nor used. The error handler has a callback function which can in principle be used to call 3rd party code. - -## Usage - -### Logger -The logger can be used similar to the streams in the C++ standard API. -To select the log level, the corresponding logger has to be used, e.g. ``LogErr``, ``LogWarn`` etc. - -``` -LogWarn() << "log message " << someValue << "log message continued"; -``` -A line break is inserted implicitly at the end (after "log message continued" in the example). - -### Error Handler -The most general use case is the following -``` -if(noError()) { - //handle the regular case -} else { - auto callback = []() { //some error handling callback}; - errorHandler(Error::kSOME_ERROR_CODE, callback, ErrorLevel::SEVERE); -} -``` - -If no callback or error level are specified, the error level is assumed to be FATAL by default. -``` -errorHandler(Error::kSOME_ERROR_CODE) -``` - -If no callback but an error level is desired, a *nullptr* has to be provided for the callback. -``` -errorHandler(Error::kSOME_ERROR_CODE, nullptr, ErrorLevel::MODERATE); -``` - -### Expects and Ensures - -Assume myAlgorithm is part of an inner API and not supposed to be called with a ``nullptr``. We may have used a reference here, this is just for illustration. -In addition the value pointed to is assumed to be in the range (-1024, 1024). While we could check this every time, this can be avoided if we specify that the caller is responsible to ensure that these conditions hold. - -``` -int32_t myAlgorithm(int32_t* ptr) { - Expects(ptr!=nullptr); - //observe the order, we only dereference after the nullptr check - Expects(*ptr > -1024 && *ptr < 1024); - - int32_t intermediate = timesTwo(*ptr); - //this may not be necessary here to ensure that the next function call is valid, - //but it states our expectations clearly - Ensures(intermediate % 2 == 0); - - int32_t result = abs(intermediate); - - Ensures(result % 2 == 0); - Ensures(result >= 0); - Ensures(result < 2048); - - return result; -} -``` -Note that in the case of ``nullptr`` checks it is also an option to use references in arguments (or ``not_null`` if it is supposed to be stored since references are not copyable). It should be considered that ``not_null`` incurs a runtime cost, which may be undesirable. -When Expects and Ensures are implemented to leave no trace in release mode, we do not incur a runtime cost using them. For this reason, it is advised to use them to document and verify assumptions where appropriate. - -### cxx::expected -This example checks the arguments and if they are valid proceeds to compute a result and returns it. -Otherwise it creates an Error object from an errorCode and returns it. - -``` -cxx::expected func(Arg arg) { - int32_t errorCode = checkArg(arg); - if(isNoError(errorCode)) { - SomeType result = computeResult(arg); - // optionally do something with result - return result; - } - return Error(errorCode); -} -``` -The caller is responsible for handling (or propagating) the error. - -``` -auto result = func(arg); -if(result.has_error()) { - auto& error = result.error(); - //handle or propagate the error -} else { - auto& value = result.value(); - //proceed by using the value -} -``` - -Alternatively a functional approach can be used. - -``` -auto successFunc = [](SomeType& value) { - //proceed by using the value -}; - -auto errorFunc = [](Error& error) { - //handle the error -}; - -func(arg).and_then(successFunc).or_else(errorFunc); -``` - -## Open Points - -### Centralized Error Handling - -It may be desirable to have a centralized error handling instance where runtime errors on application side are logged and (maybe) handled. -This could also be done in RouDi (by sending information to RouDi), but RouDi already has too much responsibility. Preferably this should be done by a separate application with this sole purpose. -If the application cannot reach the central handler, it shall try to handle the error locally if possible (at least log it). - -However, it might be too slow if this would rely on error transmission and responses. If this is to be implemented, the exact mechanism has to be decided on. - -### 3rd Party Error Handling -We need to decide how to provide an interface for 3rd party error handling, especially for the runtime. This interface will probably rely on hooks/callbacks. The signature and callsites of these need to be discussed. -This is related to centralized error handling as well. - -### Overriding Specific Error Reaction -* Do we want to provide the ability to override error reaction based on e.g. error codes? -* Do we want to disable certain error levels? (if so, this should preferably happen at compile -time with no or few runtime artifacts). It could be an option to e.g. disable all ``MODERATE`` error reaction at compile time. -* It is probably not reasonable to allow disabling reaction on fatal errors. - -This is also related to the hooks for 3rd party error handling we may want to provide. - -### Return in Case of Fatal Error -The reporting code does not need to be able to continue properly in case of a fatal error, but there needs to be a return after the error handler call. While the error handler is not required to return, it still might under certain circumstances (e.g. a mock error handler in a test case). - -The (complete) intended behavior of the error handler requires some further clarification, especially in the case of fatal errors. In the case of non-fatal errors the code invoking the error handler must be able to continue after the error handler returns. - -### Error Handling vs. Logging -Does it make sense to have ``LogErr`` without an error handler call? If an error occurs it should probably be enforced that the handler is called and not just lead to a log entry. -One reason for this is that currently it is not possible to provide an additional error message to the error handler. - -### Additional Error Information -It would be desirable to allow the possibility to provide additional messages (or even general functions/arguments) to the error handler, which is currently missing. -This can be combined with the addition of error location to the error handler. - -An optional stack-trace (at least in debug mode) may also prove very useful. -What is needed to have a limited stack-trace even in release mode? - -### Debug vs. release mode -We need to further clarify behavior in release and debug mode of the error handler and ``Expects`` and ``Ensures`` (and maybe the logger as well). Can we have a release build with additional information? (e.g. symbols for a stack-trace). - -### Assert -Do we want an Assert in addition to ``Expects`` and ``Ensures``? If so, shall it possibly be active in release mode or only debug mode? - -In principle with a sufficiently powerful Assert or ``Expects`` (resp. ``Ensures``), this should not be needed (they are equivalent in their functionality). - -## Future improvements -In this section we briefly describe ways to potentially improve or extend functionality of existing error handling components. This list is by no means exhaustive and all suggestions are up for discussion and may be refined further. - -### Logger -1. The logger could be extended to include logging over a network to a remote server or similar. -2. Support asynchronous logging. - -### Error Handler -1. Allow customization for ``MODERATE`` and ``SEVERE`` errors to continue according to a user defined configuration. -2. Add file, line and function information (using ``__FILE__``, ``__LINE__`` and ``__func__``). This would require using macros for the error handler call in a future implementation. -3. Allow generalized callbacks with variadic arguments. -4. Change the order of arguments in a future design (callback and additional arguments last). Providing the callback and potential arguments can be made fully optional this way. -5. If deactivation or reduced operation (e.g. not handling ``MODERATE`` errors) is desired, this partial deactivation should cause no (or at least very little) runtime overhead in the deactivated cases. - -### Expects and Ensures -Allow deactivation in release mode, but it should still be possible to leave them active in release mode as well if desired. Deactivation in debug mode can also be considered but is less critical. Deactivation should eliminate all runtime overhead (i.e. condition evaluation). - -### cxx::expected -1. Consider renaming ``cxx::expected`` to ``cxx::result``, which is more in line with languages such as Rust and conveys the meaning more clearly. -2. Add ``has_value`` (in addition to ``has_error``) for consistency with ``cxx::optional``. -3. Improve the monadic error handling (beyond ``and_then``, ``or_else``) to allow for better pipelining of multiple consecutive calls (especially in the success case). This requires careful consideration of supported use cases and intended behavior but can reduce control flow code (``if ... else ...``) in error cases considerably. diff --git a/doc/website/design/listener.md b/doc/website/design/listener.md deleted file mode 100644 index f8099fd7a2..0000000000 --- a/doc/website/design/listener.md +++ /dev/null @@ -1,317 +0,0 @@ -# Listener - -## Summary - -The Listener is besides the WaitSet one of the building blocks to realize a -push approach to detect and react to certain events. The Listener -offers the user the ability to attach objects with a corresponding event and -callback. Whenever the object receives the specified event the corresponding -callback is called in the Listener background thread as a reaction. - -The two key differences to the WaitSet are that the Listener is event-driven -and not event- and state-driven as the WaitSet and that the Listener creates -a separate background thread in which the event-callbacks are executed in contrast -to the WaitSet where the user has to call the event-callbacks explicitly. - -## Terminology - -- [Condition Variable](https://en.wikipedia.org/wiki/Monitor_(synchronization)#Condition_variables_2) - Used by an attached object to inform the Listener/WaitSet that an event has - occurred. -- **event** is changing the state of an object. -- **event driven** a one time reaction which is caused directly by an event. - Example: a new sample has been delivered to a subscriber. -- **state** predefined values to which the members of an object are set. -- **state driven** a repeating reaction which is continued as long as the state - persists. - Example: a subscriber has stored samples which were not inspected by the user. - -## Design - -The Listener is a variation of the -[reactor pattern](https://en.wikipedia.org/wiki/Reactor_pattern) and -the usage should be similar to the _WaitSet_ with a key difference - it should -be **event driven** and not a mixture of event and state driven, depending on -which event is attached, like in the _WaitSet_. - -### Requirements - -- Whenever an event occurs the corresponding callback should be called **once** - as soon as possible. -- If an event occurs multiple times before the callback was called, the callback - should be called **once**. -- If an event occurs while the callback is being executed the callback should be - called again **once**. -- Thread-safety: attaching an event at any time from any thread. -- Thread-safety: detaching an event at any time from any thread. - - If the callback is currently running `detachEvent` blocks until the callback - is finished. - - After the `detachEvent` call the event-callback is not called anymore, even - when the event was signalled while `detachEvent` was running and the - callback was not yet executed. - - If the callback is detached from within the event-callback then `detachEvent` - is non blocking. The event is detached right after the `detachEvent` call. -- One can attach at most one callback to a specific event of a specific object. - - Usually defined with an enum by the developer. One example is `SubscriberEvent::DATA_RECEIVED`. - - Attaching a callback to an event where a callback has been already attached - results in an error. -- One can attach the same event to different objects at the same time. -- One can attach multiple different events to a single object -- When the Listener goes out of scope it detaches itself from every attached - object via a callback provided by the attached object (like in the WaitSet). -- When the class which is attached to the Listener goes out of scope it - detaches itself from the Listener via a callback provided by the Listener - (like in the WaitSet). - -### Solution - -#### Class diagram - -``` - +---------------------------+ - | ConditionVariableData | - | - m_semaphore | - | - m_runtimeName | - | - m_toBeDestroyed | - | - m_activeNotifications | - +---------------------------+ - | 1 | 1 - | | - | 1 | n -+-----------------------------------------------+ +--------------------------------------------------+ -| ConditionListener | | ConditionNotifier | -| ConditionListener(ConditionVariableData & ) | | ConditionNotifier(ConditionVariableData &, | -| | | uint64_t notificationIndex) | -| bool wasNotified() | | | -| void destroy() | | void notify() | -| NotificationVector_t wait() | | | -| NotificationVector_t timedWait() | | - m_condVarDataPtr : ConditionVariableData* | -| | | - m_notificationIndex | -| - m_condVarDataPtr : ConditionVariableData* | +--------------------------------------------------+ -| - m_toBeDestroyed : std::atomic_bool | | 1 -+-----------------------------------------------+ | - | 1 | n - | +--------------------------------+ - | 1 | TriggerHandle | -+-------------------------------------------------+ | bool isValid() | -| Listener | | bool wasTriggered() | -| attachEvent(Triggerable, EventType, Callback) | | void trigger() | -| detachEvent(Triggerable, EventType) | | void reset() | -| | | void invalidate() | -| - m_events : Event_t[] | | void getUniqueId() | -| - m_thread : std::thread | | | -| - m_conditionListener : ConditionListener | | - m_conditionVariableDataPtr | -| | | - m_resetCallback | -| +----------------------------+ | | - m_uniqueTriggerId | -| | Event_t | | +--------------------------------+ -| | void executeCallback() | | | 1 -| | bool reset() | | | -| | bool init(...) | | | n -| | | | +-------------------------------------------------------+ -| | - m_origin | | | Triggerable (e.g. Subscriber) | -| | - m_callback | | | | -| | - m_invalidationCallback | | | void invalidateTrigger(const uint64_t triggerId) | -| | - m_eventId | | | void enableEvent(TriggerHandle&&, const EventEnum ) | -| +----------------------------+ | | void enableEvent(TriggerHandle&&) | -+-------------------------------------------------+ | void disableEvent(const EventEnum) | - | void disableEvent() | - | | - | - m_triggerHandle : TriggerHandle | - +-------------------------------------------------------+ -``` - -The Triggerable does not need to implement all `enableEvent`, -`disableEvent` variations, only the ones which are required by the use -case. The `enableEvent` and `disableEvent`, without the distincting `EventEnum`, -can be used when there is only a single event which can be triggered. - -#### Class Interactions - -- **Creating Listener:** a `ConditionVariableData` is created in the shared memory - The `Listener` uses the `ConditionListener` to wait for incoming events. - -``` - PoshRuntime -Listener | - | getMiddlewareConditionVariable : var | - | --------------------------------------> | - | ConditionListener(var) | ConditionListener - | ----------------------------------------+-----------------> | - | wait() : vector | | - | ----------------------------------------+-----------------> | -``` - -- **Attaching Triggerable Event (SubscriberEvent::DATA_RECEIVED) to Listener:** - The Listener creates a TriggerHandle and provides it to the Triggerable (Subscriber) - via `enableEvent` so that the Triggerable owns the handle. Whenever the event occurs - the Triggerable can use the `trigger()` method of the TriggerHandle to notify - the Listener. - -``` -User Listener Triggerable - | attachEvent() | | - | ------------------> | TriggerHandle | - | | create | | - | | ---------> | | - | | enableEvent(std::move(TriggerHandle)) | - | | -----------+--------------------------------------> | -``` - -- **Signal an event from Triggerable:** `TriggerHandle::trigger()` is called, the - Listener is returning from the `ConditionListener.wait()` call and retrieves - a list of all the signal notifications. The corresponding event-callbacks - are called. - -``` -Triggerable TriggerHandle ConditionNotifier ConditionListener Listener Event_t - | trigger() | | | wait() : notificationIds | | - | -------------> | notify() | | <------------------------- | | - | | -------------> | .... unblocks .... | blocks | exeuteCallback() | - | | | | | ------------------------> | - | | | | | m_events[notificationId] | -``` - -- **Triggerable goes out of scope:** The TriggerHandle is a member of the - Triggerable, therefore the d'tor of the TriggerHandle is called which removes - the trigger from the Listener via the `resetCallback` - -``` -Triggerable TriggerHandle Listener Event_t - | ~TriggerHandle | | | - | ----------------> | removeTrigger() | | - | | ----------------> | reset() | - | | via resetCallback | ------------> | -``` - -- **Listener goes out of scope:** The d'tor of the `Event_t` invalidates the - Trigger inside the Triggerable via the `invalidationCallback` - -``` -Listener Event_t Triggerable - | ~Event_t() | | - | -----------> | invalidateTrigger() | - | | -----------------------> | - | | via invalidationCallback | -``` - -#### TriggerHandle - -- **Problem:** The Triggerable should be able to notify a Listener/WaitSet - without having any knowledge about those class so that circular - dependencies can be prevented. Furthermore, the Triggerable must be able - to remove its attached events when it goes out of scope. -- **Solution:** The dependency inversion principle, create an abstraction which - is known by both, the TriggerHandle. Created by the Listener/WaitSet and - attached to the Triggerable so that it can notify the Listener/WaitSet via - the underlying `ConditionNotifier` with `TriggerHandle::notify()`. - The cleanup task is performed by the `m_resetCallback` so that the - Triggerable has no dependencies to any Notifyable. - -#### Condition Variable - -The `ConditionListener` and `ConditionNotifier` are two different interfaces to -the same class which state is stored in the `ConditionVariableData` class. The -intention of the separation is to provide one side (e.g. Triggerable) only -an API to notify the Notifyable (e.g. Listener) whereas the Notifyable can only -wait on events. So the contract is reflected in the design. - -- **Problem:** Since the Listener reacts on events and not states it requires - the knowledge by whom it was notified. -- **Solution:** - - Every TriggerHandle has a unique id which is used as index in `ConditionNotifier`. - - When `ConditionNotifier::notify` is called the Listener is informed via the - `NotificationVector_t` return value from `ConditionListener::wait()` which - index notified him. Hence it is the same as the unique id of the TriggerHandle - the Listener knows which Triggerable notified him. - -#### Event_t - -##### Concurrency - -The Listener must be able to attach, detach events concurrently. Additionally, -it supports that a callback can attach or detach further events or detach its -corresponding event concurrently. Furthermore, the Listener supports that a -callback is called concurrently while events are being attached/detached. - -To realize this we created the `Event_t` abstraction which is stored in an -array called `m_events`. If we would like to attach or detach an event we either -initialize `Event_t::init()` or reset `Event_t::reset()` the corresponding entry -in the `m_events` array. The array has the advantage that the data structure -itself never changes during runtime therefore it does not have to be thread-safe. - -The thread-safety must then be ensured by the `Event_t` class itself. Since -every concurrent action is contained in `Event_t` we can use -`concurrent::smart_lock` in combination with a `std::recursive_mutex` to -guarantee the thread-safe access. - -1. Concurrent attach/detach event and callback execution, ensured by - thread-safe `Event_t`. -2. Detaching itself from within a callback is ensured via the - `std::recursive_mutex`. -3. Attaching/detaching arbitrary events from within a callback is ensured by - securing every `Event_t` object in `m_events` with a `concurrent::smart_lock`. - This would not be possible if the data structure itself had to be thread-safe. - -##### Lifetime - -Since the event contains everything which is required to handle events it has -the responsibility to ensure the life-time of the TriggerHandle. This is done -by the `m_invalidationCallback` which is called in `Event_t::reset()` to -invalidate the TriggerHandle in the corresponding Triggerable. This is either -done when an event is detached or the Listener goes out of scope. - -#### Triggerable (e.g. Subscriber) - -The Triggerable is a set of classes of which events can be attached to the Listener. - -It is possible to either attach a specific event of a class to the Listener or -attach the class without providing an event. - -The basic idea is that the Listener creates a TriggerHandle whenever an event -is attached and provides that TriggerHandle to the corresponding Triggerable. -The Triggerable uses then the TriggerHandle to notify the Listener about events. - -##### Triggerable With Single Event - -Every Triggerable requires: - -1. The private methods: - -```cpp -void enableEvent(iox::popo::TriggerHandle&& triggerHandle) noexcept; -void disableEvent() noexcept; -void invalidateTrigger(const uint64_t uniqueTriggerId) noexcept; -``` - -##### Triggerable With Multiple Events - -Every Triggerable requires: - -1. An `enum class` which uses `iox::popo::EventEnumIdentifier` as underlying type. - -```cpp -enum class EventEnum : iox::popo::EventEnumIdentifier { - EVENT_IDENTIFIER, - ANOTHER_EVENT_IDENTIFIER, -}; -``` - -2. The private methods: - -```cpp -void enableEvent(iox::popo::TriggerHandle&& triggerHandle, const EventEnum event) noexcept; -void disableEvent(const EventEnum event) noexcept; -void invalidateTrigger(const uint64_t uniqueTriggerId) noexcept; -``` - -The methods above are used by the Listener to transfer the ownership of the TriggerHandle -to the Triggerable. The Triggerable should have one TriggerHandle member for every -attachable event/state. -The TriggerHandle is then used to notify the Listener by the Triggerable that -a certain event has occurred. - -3. It must be friend with `iox::popo::NotificationAttorney`. It is possible to - provide public access to the previous methods but then the user has the ability - to call methods which should only used by the Listener. - - diff --git a/doc/website/design/lockfree_queue.md b/doc/website/design/lockfree_queue.md deleted file mode 100755 index 251165565d..0000000000 --- a/doc/website/design/lockfree_queue.md +++ /dev/null @@ -1,135 +0,0 @@ -Remark: This is a preliminary description and will be reworked when the capacity feature is implemented. - -# Lock Free queue - -We explain some details of the lock free queue in order to provide an intuition for the way the lock free queue works. This could serve as a basis for a formal proof later if desired. - -## Index queue analysis - -In the following all numbers are natural numbers and hence bounded unsigned integers in the implementation. - -### Capacity and Cycle Length -The index queue has some capacity n and stores indices in the range [0, n[. -The number n is also called cycle length. - -Assume n = 4 in this explanation, so we can store the indices 0, 1, 2, 3. - -### Index representation -Each index i corresponds to an equivalence class [i] modulo n, i.e. -[i] = {j | j = c*n + i, c >= 0} - -This means j = 3, 7, 11, ... all represent the same index 3 (pointing to the last element in the array) - -Given j, we have i = j % n and c = j / n (where % is modulo and / integer division). - -Another way to represent such a value j uniquely is as a pair (c,i) and -we call this a cyclic index with cycle c and index i. - -This is done for two reasons: detection of empty queues and to eliminate the ABA problem for all practical scenarios. -It is imperative that such indices can be used in compare and swap operations (CAS), i.e. -will not exceed 64 bits on standard architectures supporting 64 bit CAS. -Therefore j can be assumed to be an unsigned 64 bit integer. - -### Queue representation -The queue has a head H, tail T and an array of n values. All of them are cyclic indices as in 2. -Initially the queue is empty and head and tail point to index 0 but have both cycle 1. -Array elements are 0 and represented as (0,0). - -Initially empty queue -``` -[ (0,0), (0,0), (0,0), (0,0) ] H=(1,0) T=(1,0) -``` -Initially full queue with all indices -``` -[ (0,0), (0,1), (0,2), (0,3) ] H=(0,0) T=(1,0) -``` - - -### Head and Tail monotonicity -We always insert (push) at Tail and remove (pop) at Head. -Head and Tail both increase strictly monotonic (making ABA problems unlikely). -I.e. a push causes tail to increase by one (causing the cycle to increase after n pushes) and similarly each pop causes Head to increase by one. - -### Push operation -push(y) causes the cycle at the position Tail points to to be replaced with Tails cycle while the element there is replaced with x -via a CAS operation. -Afterwards Tail is increased by 1 (potentially not immediately, but before the next push takes effect). -``` -[ (c,?), (c,x), (c-1,?), (c-1,?) ] H=(c,1) T=(c,2) -``` -push(y) -``` -[ (c,?), (c,x), (c,y), (c-1,?) ] H=(c,1) T=(c,3) -``` - -We only push if the cycle at the element Tail points to is exactly one behind Tails cycle. - -Constraint: we can never push more than n elements (and our use case does not require this). - -### Pop operation -Pop reads the value at the index of head and returns it if the cycle matches Heads cycle and a CAS to increase head by 1 succeeds. - -``` -[ (c,?), (c,y), (c,x), (c-1,?) ] H=(c,1) T=(c,3) -``` -pop returns y -``` -[ (c,?), (c,x), (c,y), (c-1,?) ] H=(c,2) T=(c,3) -``` - -If the cycle is behind Heads cycle the queue is empty (at the time of this check) and nothing is returned. - - -### Head Tail relationship -Head is always at most Tail, i.e. H <= T. (Technically we only have H - 1 <= T in general, cf. push implementation) - -The general situation is one of the following, with c indicating the cycle of the values and Head and Tail. - -(i) Head and Tail are at cycle c and ***** marks the region in the array with values logically in the queue. -``` -[ c |*****c*****| c-1 ] - H = (c,i) T= (c,j) -``` - -(ii) Head is one cycle behind Tail. -``` -[******c*****| c-1 |******c-1*****] - T = (c,i) H= (c-1,j) -``` - -(iii) Empty queue -``` -[ c | c-1 ] - T = H = (c,i) -``` -### ABA prevention - -Note that the monotonicity of Head and Tail combined with suitable CAS make ABA problems not a practical concern. -Moreover, even if we require many bits for the index and thus have few bits for the cycle, we still require a complete uint64 -wraparound for an ABA problem to occur (in addition to re-inserting the same value). - -This is simply because if the maximum cycle is small then the cycle length n must be large and -we increase the cycle every n pushes (or pops), but the numbers representing the index increase by one every operation. - -### Lock free analysis -Claim: -The queue is operation-wise lock free: pushes cannot block pops arbitrarily and vice versa. -Furthermore, between concurrent pushes and pops one of each type always succeeds in a finite amount of time. - -Proof sketch: -- only pushes read/write Tail but pops do not -- pushes modify array values which pop reads and compares (CAS) - but we can have at most n pushes before this stops (push constraint in 5.) and then pop will succeed without interference from push -- only pops read/write/CAS Head -- claim follows now from loop/read/write/CAS structure in implementation - -Therefore pushes and pops do not interfere to block each other. - -However, pushes can block other pushes and pops other pops. -In particular operation-wise lock freedom implies the queue is lock free. - -Note that there is no fairness guarantee. In principle the same push thread or pop thread might always succeed but in practice this is unlikely. -The queue is therefore not wait free. - -This could technically be mitigated somewhat with a more complex logic keeping track of operation failures -but seems not to be worth it in practice. (incomplete analysis, further studies required) diff --git a/doc/website/design/relocatale_pointer.md b/doc/website/design/relocatale_pointer.md deleted file mode 100644 index 833311b4f0..0000000000 --- a/doc/website/design/relocatale_pointer.md +++ /dev/null @@ -1,215 +0,0 @@ -# Relocatable and Relative Pointers - -When memory is shared between applications and each application maps the shared memory to a different starting address -in its own address space, raw pointers cannot be used anymore. This is due to the fact that a pointer pointing to some -object in shared memory in one address space will in general not point to the same object in another address space (or -even to the shared memory), i.e. the pointer is invalid. In this case, different pointer concepts such as relative and -relocatable pointers must be used. Depending on the shared memory setup and the relationship between pointer and pointee, -there are different use cases which we need to cover. - -Both RelativePointer and RelocatablePointer are template pointer classes that try to retain much of the normal pointer -syntax where possible while still being valid across address space boundaries. This means that operations like -assignment, dereferencing, comparison and so on work similar to raw pointers. They do not possess reference counting -semantics in this implementation, but extensions for this functionality are possible. - -## Shared Memory Setup - -In general, we have many shared memory segments, each starting at some address and containing a specified number of -bytes. Specifically we distinguish between management and payload segments. The management segment contains metadata -used by RouDi and the runtime to communicate, store common structures such as ports and so on. The payload segment -contains the actual data samples sent by senders in some application to receivers in other applications. - -## Relocatable pointer - -When pointer and pointee are located in the same shared memory segment, the lightweight (compared to a relative pointer) -relocatable pointer can be used. That is, we have the following scenario. - -Pointer p points to object X of type T and both are stored in shared memory segment S. - -``` -Shared Memory S: p X - |____________________^ - -App1 a1 b1 c1 -App2 a2 b2 c2 -``` - -Let a1, b1, c1 and so on be the addresses of segment S, pointer p and object X in application 1 and similarly a2, b2 and -c2 in application2. If application 2 maps the memory differently they will be shifted by some common offset d depending -on the individual memory mapping: - -a2 = a1 + d, b2 = b1 + d, c2 = c1 + d - -This is why storing a raw pointer to X will not be sufficient, the value c1 of p will not point to X in application 2. -However, storing the difference between the location of p and X will work since it is an invariant in both address -spaces. - -Thus, instead of using a raw pointer p, we just use a relocatable pointer RelocatablePointer pointing to X. This pointer -can then be dereferenced in both address spaces and will point to X. - -Note that if we only use relocatable pointers in this way in complex data structures that rely on pointers (such as -lists and trees) in shared memory, we can in fact copy the whole shared memory segment somewhere else (i.e. *relocate* -it, hence the name) without compromising the data structure integrity. That is, the list elements in the copy correctly -point to their successor elements in the copied memory. This also presupposes that we do not rely on other side effects -of deep copying (i.e. nontrivial copy constructors). - -### Construction and Assignment - -Assume we have an object X of type T in shared memory. Then we can construct relocatable pointers p in the *same -segment* (e.g. as members of objects that reside in this segment or via emplacement new at a memory address in this -segment). - -default construct a logical nullptr -+ `RelocatablePointer p;` - -construct relocatable pointing to X -+ `RelocatablePointer p(&X);` - -assign relocatable to point to X -+ `p = &X;` - -### Operations -Most operations for raw pointers also work for relocatable pointers. In particular, a RelocatablePointer is compatible -with a raw pointer T* in many cases, such as assignment, to provide seamless integration and convenient syntax. - -comparison with raw pointers -+ `if(p != nullptr) { \\do something }` - -+ `T* raw = &X; if(p == raw) { \\do something }` - -assignment to raw pointer (resolves the pointer in the current address space) -+ `T* raw = p;` - -explicitly get the raw pointer -+ `T* raw = p.get();` - -dereference -+ `T& t = *p;` - -access members -+ `p->func();` - -### Performance - -Since relocatable pointers just measure the distance to the pointee from itself it just requires a number to store this -pointer difference. This number has the same size as a pointer itself, i.e. 64 bit on 64 bit architectures. All -operations use just a few simple arithmetic instructions in addition to pointer dereferencing and hence they do incur -little overhead compared to raw pointers. - -## Relative pointer - -When pointer and pointee are located in different shared memory segments, it gets more complicated and relocatable -pointers are no longer appropriate. This happens in our setup with metadata in the management segment referencing data -in the payload segment. - -Pointer rp is stored in segment S1 and points to object X of type T in segment S2. - -``` -Shared Memory S1: rp S2: X - |_______________________^ - -App1 a1 b1 c1 d1 -App2 a2 b2 c2 d2 -``` - -Now it is no longer true in general that both segments will be offset by the same difference in App2 and therefore -relocatable pointers are no longer sufficient. - -Relative pointers solve this problem by incorporating the information from where they need to measure differences (i.e. -*relative* to the given address). This requires an additional registration mechanism to be used by all applications -where the start addresses and the size of all segments to be used are registered. Since these start address may differ -between applications, each segment is identified by a unique id, which can be provided upon registration by the first -application. In the figure, this means that the starting addresses of both segments (a1, a2 and c1, c2) would have to be -registered in both applications. - -Once this registration is done, relative pointers can be constructed from raw pointers similar to relocatable pointers. -However, it should be noted that relocating a memory segment will invalidate relative pointers, i.e. relative pointers -are NOT relocatable. This is because the registration mechanism cannot be automatically informed about the -copy of a whole segment, such a segment would have to be registered on its own (and the original segment deregistered). - -### Registration - -register the start pointer start1 of a segment with a specific size (this happens once after the memory is mapped) -+ `auto id = registerPtr(start1, size);` - -register the start pointer of the segment start2 in another application with a specific id (again, this happens after -the other application maps the memory) -+ `registerPtr(id, start2, size);` - -unregister specific id (also invalidates relative pointers using this id, they cannot be used safely after this -operation anymore) -+ `unregisterPtr(id);` - -unregister all ids (also invalidates all relative pointers) -+ `unregisterAll();` - - - -### Construction and Assignment - -Assume we have an object X of type T in shared memory. Then we can construct relative pointers rp pointing to *arbitrary -registered segments*. - -default construct a logical nullptr -+ `RelativePointer rp;` - -construct relative pointer pointing to X (provided the segment containing X was registered) -+ `RelativePointer rp(&X);` - -assign relative pointer to point to X -+ `rp = &X;` - -copy construction -+ `RelativePointer rp2(rp);` - -copy assignment -+ `RelativePointer rp2 = rp;` - -### Operations - -As for relocatable pointers, operations for raw pointers also work for relative pointers. - -comparison with raw pointers -+ `if(rp != nullptr) { \\do something }` - -+ `T* raw = &X; if(p == raw) { \\do something }` - -assignment to raw (resolves the pointer in the current address space) -+ `T* raw = rp;` - -dereference -+ `T& t = *p;` - -implicitly get the raw pointer -+ `T* raw = rp;` - -explicitly get the raw pointer -+ `T* raw = rp.get();` - -access members -+ `p->func();` - -### Performance - -A relative pointer measures the distance to the pointee relatively to a registered address and therefore needs to store two -numbers, one for the distance itself and the other one to identify the segment base address to measure from. For this reason, -the operations incur slightly more overhead compared to relocatable pointers. - -## Atomic usage - -There is a technical problem using both RelocatablePointer and RelativePointer as a type in a std::atomic. This is essentially -impossible as an atomic requires its type to be copyable/movable (to be loaded) but on the other hand, this copy -constructor must be trivial, i.e. performable with a shallow memcpy. Therefore, the types used in atomic cannot implement -custom copy/move. This is not possible for RelocatablePointer and RelativePointer as both require operations performed -during copying, which cannot be done by a simple shallow copy. - -To support the use case of an atomic relocatable pointer, the template AtomicRelocatablePointer is provided. It is a -basic version of a RelocatablePointer, which lacks some of its move functionality and other operations. However, its -existing operations behave like RelocatablePointer but in an atomic way (but also incur additional overhead caused by -atomic loads and stores). Therefore, this object can be used concurrently by multiple threads, in contrast to -RelativePointer and RelocatablePointer. This atomic version also utilizes lock free atomic operations internally (when -available on the target architecture) as the data to be stored is just a 64 bit word. - -Note that this cannot be done as easily for RelativePointer as it requires more than 64 bit to store its internal -information. A construction where atomicity is guaranteed via locking could be provided in the future. - diff --git a/doc/website/design/request_response_communication.md b/doc/website/design/request_response_communication.md deleted file mode 100644 index e268a5da98..0000000000 --- a/doc/website/design/request_response_communication.md +++ /dev/null @@ -1,131 +0,0 @@ -# Request/Response Communication - -## Summary - -Beside the publish-subscribe messaging pattern initially supported by iceoryx, -the request-response pattern is also widely used in the field where iceoryx is headed. -Since we aim for a good integration with ROS 2, this is also necessary to support its -`Service` communication method which is often used to control robots. - -While publish-subscribe is a data distribution pattern which continuously delivers -updates on its data, request-response is a remote procedure call and task distribution pattern -which delivers data only on demand. This means there is one or several clients -requesting a server to perform a task. - -In iceoryx, the client and server will be built on top of the same building blocks -we already use for publish-subscribe, like the `ChunkSender` and `ChunkReceiver`. - -## Terminology - -| Name | Description | -| :---------------- | :------------------------------------------------------- | -| Request-Response | communication pattern which delivers data on demand | -| Publish-Subscribe | communication pattern which continuously delivers updates on its data | -| RPC | a remote procedure call to offload task to a server | -| Client | an entity invoking RPCs | -| Server | an entity processing RPCs | - -## Design - -### Considerations - -Client and server shall be able to be used in combination with the `WaitSet` and `Listener` -to provide a means of event based communication instead of polling. - -Like with publish-subscribe, the `ServiceDescription` will be used to connect clients with a corresponding server. - -In order to support asynchronous requests, a sequence ID should be part of each request which will be copied to the corresponding response. - -### Solution - -This is an overview of the untyped `Client` and `Server` classes. - -![simple class diagram](overview_class.svg) - -The `Client` and `Server` are reusing the `ChunkSender` and `ChunkReceiver` building blocks. The `Client` uses a `ChunkSender` to send requests and a `ChunkReceiver` to get the responses while the `Server` uses a `ChunkReceiver` to get the requests and a `ChunkSender` to send the responses. - -#### Typed API - -![typed API](typed_api.svg) - -Since the `Response` is tied to a specific `Request`, the `loan` method takes a `Request` to populate the `Response` with the correct settings. -Alternatively, a `Request` could have a `createResponse` method which returns a `Response` with the correct settings. - -#### Untyped API - -![untyped API](untyped_api.svg) - -Similar to the the typed API, `loan` takes a pointer to a `RequestHeader` to populate the `ResponseHeader` with the correct settings. - -#### Client Port - -![client port](client_port.svg) - -The `ClientPortData` is located in the shared memory and contain only the data but no methods to access them. -`ClientPortUser` is the class providing the methods for the user access and `ClientPortRouDi` provides the -interface RouDi needs to connect the client to the server and to cleanup the port resources. - -#### Server Port - -![server port](server_port.svg) - -Similar to the Client Port, the Server Port has `ServerPortData` which is located in the shared memory and contain only the data but no methods to access them. -`ServerPortUser` is the class providing the methods for the user access and `ServerPortRouDi` provides the -interface RouDi needs to connect the client to the server once the server offers its service and to cleanup the port resources. - -It must be ensured that only one server with a given `ServiceDescription` can run at a time. - -#### Request/Response Header - -![rpc header](request_response_header.svg) - -Since request and response need to encode different meta-information, we also need different header for the messages. -The common data is aggregated in `RpcBaseHeader` which contains a `cxx::UniqueId` to the `ClientChunkQueueData_t` and a sequence ID. -The `cxx::UniqueId` is used to identify the queue which receives the response and used as identifier for `ChunkDistributor::deliverToQueue`. -This method will iterate over all stored queues and matches the `ChunkQueueData::m_uniqueIdOfOwner`. -As optimization, the `lastKnownQueueIndex` will be used to do a fast lookup and downgrades to a full lookup if the queue IDs don't match. -```cpp -class ChunkDistributor { - ... - cxx::expected deliverToQueue(cxx::UniqueId uniqueQueueId, uint32_t lastKnownQueueIndex, mepoo::SharedChunk chunk); - ... -}; -``` - -The sequence ID is used to match a response to a specific request if multiple requests are invoke asynchronously. -Depending on the client and server options, a request might be dropped or a server could have a worker pool -which results in sending the responses in a different order than the request were received. -The sequence ID must be set by the user and also checked by the user on response. - -The `RequestHeader` has also the option to specify a message as fire and forget, which means it won't get a response to this request. - -#### Client/Server Options - -![client and server options](client_and_server_options.svg) - -The client and server options can be used to control certain aspects of the clients and servers. -Beside setting the capacity of the queues and defining whether a client should be connected and a server offering on creation, -the behaviour for a slow client and server can be defined. -A client can ask a server to block if its response queue is full. -The server will obey if the corresponding `clientTooSlowPolicy` is set to `ConsumerTooSlowPolicy::WAIT_FOR_CONSUMER`. -If the options don't match, the client will not be connected to the server, similar to the behavior of publisher and subscriber. - -#### Client Service Discovery - -The client is guided by the following state machine. - -![client state machine](client_state_machine.svg) - -Similar to the subscriber state machine, the client passes it's response queue with the `CONNECT` CaPro message to the server. -The server will pass its request queue with the `ACK` CaPro message to the client. - -Following is a sequence diagram which shows all this cases - -![client and server service discovery](client_and_server_service_discovery.svg) - -### Code example - -## Open issues - -- integration into a gateway, e.g. the DDS gateway -- check if a Listener can be used for a timed wait for a response on the client side diff --git a/doc/website/design/service-discovery.md b/doc/website/design/service-discovery.md deleted file mode 100644 index 7614ea079c..0000000000 --- a/doc/website/design/service-discovery.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: Service discovery ---- - -{! ../iceoryx/doc/design/service-discovery.md !} diff --git a/doc/website/tutorials/.pages b/doc/website/examples/.pages similarity index 94% rename from doc/website/tutorials/.pages rename to doc/website/examples/.pages index 5bfa60c135..b222d1410a 100644 --- a/doc/website/tutorials/.pages +++ b/doc/website/examples/.pages @@ -3,6 +3,7 @@ nav: - icehello.md - icedelivery.md - icedelivery_in_c.md + - icedocker.md - iceoptions.md - callbacks.md - callbacks_in_c.md diff --git a/doc/website/tutorials/callbacks.md b/doc/website/examples/callbacks.md similarity index 100% rename from doc/website/tutorials/callbacks.md rename to doc/website/examples/callbacks.md diff --git a/doc/website/tutorials/callbacks_in_c.md b/doc/website/examples/callbacks_in_c.md similarity index 100% rename from doc/website/tutorials/callbacks_in_c.md rename to doc/website/examples/callbacks_in_c.md diff --git a/doc/website/tutorials/complexdata.md b/doc/website/examples/complexdata.md similarity index 58% rename from doc/website/tutorials/complexdata.md rename to doc/website/examples/complexdata.md index 1399e94302..455bdce629 100644 --- a/doc/website/tutorials/complexdata.md +++ b/doc/website/examples/complexdata.md @@ -2,5 +2,4 @@ title: Sending/receiving some of the iceoryx STL container surrogates --- -{! ./../iceoryx_examples/complexdata/README.md !} - +{! ../iceoryx/iceoryx_examples/complexdata/README.md !} diff --git a/doc/website/tutorials/ice_access_control.md b/doc/website/examples/ice_access_control.md similarity index 100% rename from doc/website/tutorials/ice_access_control.md rename to doc/website/examples/ice_access_control.md diff --git a/doc/website/tutorials/icecrystal.md b/doc/website/examples/icecrystal.md similarity index 100% rename from doc/website/tutorials/icecrystal.md rename to doc/website/examples/icecrystal.md diff --git a/doc/website/tutorials/icedelivery.md b/doc/website/examples/icedelivery.md similarity index 100% rename from doc/website/tutorials/icedelivery.md rename to doc/website/examples/icedelivery.md diff --git a/doc/website/tutorials/icedelivery_in_c.md b/doc/website/examples/icedelivery_in_c.md similarity index 100% rename from doc/website/tutorials/icedelivery_in_c.md rename to doc/website/examples/icedelivery_in_c.md diff --git a/doc/website/tutorials/icedocker.md b/doc/website/examples/icedocker.md similarity index 100% rename from doc/website/tutorials/icedocker.md rename to doc/website/examples/icedocker.md diff --git a/doc/website/tutorials/iceensemble.md b/doc/website/examples/iceensemble.md similarity index 100% rename from doc/website/tutorials/iceensemble.md rename to doc/website/examples/iceensemble.md diff --git a/doc/website/tutorials/icehello.md b/doc/website/examples/icehello.md similarity index 100% rename from doc/website/tutorials/icehello.md rename to doc/website/examples/icehello.md diff --git a/doc/website/tutorials/iceoptions.md b/doc/website/examples/iceoptions.md similarity index 100% rename from doc/website/tutorials/iceoptions.md rename to doc/website/examples/iceoptions.md diff --git a/doc/website/tutorials/iceperf.md b/doc/website/examples/iceperf.md similarity index 100% rename from doc/website/tutorials/iceperf.md rename to doc/website/examples/iceperf.md diff --git a/doc/website/tutorials/index.md b/doc/website/examples/index.md similarity index 68% rename from doc/website/tutorials/index.md rename to doc/website/examples/index.md index 7c077be205..f295b64714 100644 --- a/doc/website/tutorials/index.md +++ b/doc/website/examples/index.md @@ -1,5 +1,5 @@ --- -title: List of examples +title: Examples --- {! ../iceoryx/iceoryx_examples/README.md !} diff --git a/doc/website/tutorials/singleprocess.md b/doc/website/examples/singleprocess.md similarity index 100% rename from doc/website/tutorials/singleprocess.md rename to doc/website/examples/singleprocess.md diff --git a/doc/website/tutorials/user_header.md b/doc/website/examples/user_header.md similarity index 60% rename from doc/website/tutorials/user_header.md rename to doc/website/examples/user_header.md index 8245d2ce1c..0246ee19fb 100644 --- a/doc/website/tutorials/user_header.md +++ b/doc/website/examples/user_header.md @@ -2,4 +2,4 @@ title: Using a user-header for additional meta-information like timestamps --- -{! ../iceoryx_examples/user_header/README.md !} +{! ../iceoryx/iceoryx_examples/user_header/README.md !} diff --git a/doc/website/tutorials/waitset.md b/doc/website/examples/waitset.md similarity index 100% rename from doc/website/tutorials/waitset.md rename to doc/website/examples/waitset.md diff --git a/doc/website/tutorials/waitset_in_c.md b/doc/website/examples/waitset_in_c.md similarity index 100% rename from doc/website/tutorials/waitset_in_c.md rename to doc/website/examples/waitset_in_c.md diff --git a/doc/website/getting-started/overview.md b/doc/website/getting-started/overview.md index 3c475cd159..72d9195759 100644 --- a/doc/website/getting-started/overview.md +++ b/doc/website/getting-started/overview.md @@ -57,7 +57,7 @@ and exhaust memory. We have to handle this potential error since the expected cl attached. This means we get a warning (or an error when build in strict mode) when we don't handle it. We could also explicitly discard it with `IOX_DISCARD_RESULT` which is discouraged. If you want to know more about `expected`, take a look at -[How optional and error values are returned in iceoryx](../advanced/how-optional-and-error-values-are-returned-in-iceoryx.md). +[How optional and error values are returned in iceoryx](how-optional-and-error-values-are-returned-in-iceoryx.md). Let's create a corresponding subscriber. @@ -66,8 +66,8 @@ iox::popo::Subscriber subscriber({"Group", "Instance", "CounterTop ``` Now we can use the subscriber to receive data. For simplicity, we assume that we periodically check for new data. It -is also possible to explicitly wait for data using the [WaitSet](../examples/waitset/) or the -[Listener](../examples/callbacks/). The code to receive the data is the same, the only difference is the way we wake +is also possible to explicitly wait for data using the [WaitSet](waitset.md) or the +[Listener](callbacks.md). The code to receive the data is the same, the only difference is the way we wake up before checking for data. ```cpp @@ -117,6 +117,8 @@ Afterwards, we can start the applications which immediately connect to the RouDi When the application terminates, the runtime cleans up all resources needed for communication with RouDi. This includes all memory chunks used for the data transmission which may still be held by the application. +Before going into the details in the next section, the following animations depicts the course of events. + ![Overview](https://user-images.githubusercontent.com/8661268/74612998-b962bc80-510a-11ea-97f0-62f41c5d287b.gif) We now briefly define the main entities of an iceoryx system which were partially already used in the example above. @@ -256,7 +258,7 @@ picture below. This is sufficient for simple use cases but inefficient in genera latency and wake-ups without data. An alternative approach to receive data is to wait for user-defined events to occur. This is provided by our `WaitSet` and `Listener` which are introduced in the following sections. -![Polling alternatives](../images/avoid-polling.svg) +![Polling alternatives](avoid-polling.svg) ### WaitSet @@ -298,12 +300,12 @@ For more information about the Listener see our ## API The API is offered in two languages, C++ and C. Detailed information can be found in the -[C++ example](../examples/icedelivery/) and -[C example](../examples/icedelivery_in_c/). +[C++ example](icedelivery.md) and +[C example](icedelivery_in_c.md). Many parts of the C++ API follow a functional programming approach which is less error-prone. This requires using the monadic types `cxx::expected` and `cxx::optional` which are introduced -[here](../advanced/how-optional-and-error-values-are-returned-in-iceoryx.md). +[here](how-optional-and-error-values-are-returned-in-iceoryx.md). With the C++ API, we distinguish between the `typed API` and the `untyped API`. In the typed API, the underlying data type is made apparent by typed pointers or references to some data type T (often a template parameter). This allows diff --git a/doc/website/getting-started/what-is-iceoryx.md b/doc/website/getting-started/what-is-iceoryx.md index 3d79aca642..6cc49dc430 100644 --- a/doc/website/getting-started/what-is-iceoryx.md +++ b/doc/website/getting-started/what-is-iceoryx.md @@ -11,7 +11,6 @@ consumers of messages. On its own, this is not a new innovation as the approach However, iceoryx takes the approach further, ending up in an inter-process-communication technology with a publish/subscribe architecture that is fast, flexible and dependable. - ## Fast With the iceoryx API, a publisher can write the message directly into a chunk of memory that was previously requested @@ -21,11 +20,11 @@ zero-copy — an end-to-end approach from publishers to subscribers without crea Avoiding the copies on API level is crucial when GBytes of sensor data have to be processed per second on robotics and autonomous driving systems. Therefore the iceoryx team contributed to the standardization of true zero-copy capable -APIs in [ROS 2](https://www.ros.org/) and [AUTOSAR Adaptive](https://www.autosar.org/standards/adaptive-platform/). - +APIs in [ROS 2](https://www.ros.org/) and [AUTOSAR Adaptive](https://www.autosar.org/standards/adaptive-platform/). + On modern processors iceoryx has a latency of less than 1 µs for transferring a message. And the best message is that this latency is constant as size doesn't matter. Want to give it a try? Then have a look at our -[iceperf example](../examples/iceperf) after having made the first steps. +[iceperf example](iceperf.md) after having made the first steps. ## Flexible @@ -36,13 +35,13 @@ integrating iceoryx as shared memory backbone into a bigger framework. The APIs support polling access and event-driven interactions with the [Waitset](../overview/#waitset) and [Listener](../overview/#listener). Applications can be started and stopped flexibly as there is a service discovery -behind the scenes that dynamically connects matching communication entities. +behind the scenes that dynamically connects matching communication entities. That iceoryx has the right set of features can be seen from the already existing integrations in middleware and frameworks such as [Eclipse Cyclone DDS](https://github.com/eclipse-cyclonedds/cyclonedds), [eCAL from Continental](https://continental.github.io/ecal/), [RTA-VRTE from ETAS](https://www.etas.com/en/products/rta-vrte.php) and - [Apex.OS from Apex.AI](https://www.apex.ai/apex-os). + [Apex.OS from Apex.AI](https://www.apex.ai/apex-os). ## Dependable @@ -52,13 +51,13 @@ design and implementation of features. The usage of heap, exceptions and any und to increase the predictability. Instead a custom memory allocation is being used, based on static memory pools. Additionally, the handling of return values and error cases was inspired by upcoming C++ features and other languages like Rust (details can be found -[here](../../advanced/how-optional-and-error-values-are-returned-in-iceoryx/)). +[here](how-optional-and-error-values-are-returned-in-iceoryx.md)). As different processes are operating on shared data structures, avoiding deadlocks is becoming all the more important. iceoryx uses lock-free data structures like the multi-producer multi-consumer (MPMC) queue that was written portably -thanks to modern C++. +thanks to modern C++. The tools available for automotive-compliant software development are always one or two releases behind the latest C++ standard. This fact, combined with our already mentioned constraints, led to a bunch of STL like C++ classes that have the goal to combine modern C++ with the reliability needed for the domains iceoryx is used in. They can be found in -the iceoryx hoofs which are introduced [here](../../advanced/iceoryx_hoofs/) +the iceoryx hoofs which are introduced [here](iceoryx_hoofs.md). diff --git a/doc/website/images/favicon.ico b/doc/website/images/favicon.ico index 2e89a3c7b6aa79c6b15da1528a129594ac92b6b2..6c7d7e8636816cc781d21a1723ff7c18ae5e17f3 100644 GIT binary patch literal 15406 zcmeHN2~?BE7XIC8>r<(15o<+!f(sxngq?)21)?kp7+Dlhf-K6uN(clp9Fhb7uKx?)_%|nS1Bn83dVy z%t5~V5&`Kd#Az;qC?N>4YSka*^-B;W6Vz?q{O9+z2y#{wLDqvdpbO~nM?FXc20VH* z=mE%&F%SrZIh9BJhy(-x9b>2~^k3E46qD*_y+wMAB#@5AXp7<))5a7(jN>RYW{>g> zNhfR5D>=mDRoSHP!X1@l4*O%s_7+;Q+&(TQvYWVherddy+V1`ic6W0Hskkf>|F$A} z>-+2w{IRC0u+UV7g9!NU%&#A;!{epqr$ks-KE0DSbpI;nRzW!KN7(;p6#jiqxMAJR z^GWR`N!AQEQ|UGQXha`W*6e5qZ+v$5$a@&e63km*|5Cc%t4xY+nbR)S4L{`gqc?68 z`xn3du&yNCeck0VEVus7lHuWPYEMco3{`;ex9xe*T^A!}TrYKGzGWy1!>rv^$N} zzFzbZDsx@UWhLW1Rq&hutZRCZPEu*IeNAqNPBY^mO70_l=WRtVUrVu5*f28MZj)FB z`i*&?eT3(fe1>vn6^NQ3LizLw0m_Im%ZX2*|IsEUIC&Ekg3SA9!wG8OK+qLAa@ai) zi?Z%Y%bebt)(XXH~}lO@S}|{%--G|*yG66SO_vzo7-jq7wW z+q{)R+_Gn~QBVV|B!0ZYd79653@M@Oy`wrP3e*o1g{?Hhdsh+1EfnaPp;Ctakc06W zVLnzeo0ELd z5xg{oDTfeItZV1uTOt2Pec3|{51mjh-;pWX2UAs<=SS^nOYBc&I-nsBwA4b;bh2i@ zF#LGidf4e2JnqjK;+_}4nJM8&nC`QYykQ=mhvL1_C3Pj_#DNAUmVV)hfiw^fCKMXk(?MJ)y|BXqY+iTYr20B;d z`qQzh>o*9?-*cCoN-4)Bf|oZecQ?l#%?j4K$>k5{9+U@YP71yZvc*oU;yMLk`SEK{ z_9d)4UKZqc@lmFF6o}3s*Vs}$a*OS|ML#eUO z4BRN^S$@1eR`>0<$N~=2Nx!+4#=O)Ldnt=$AAQJH-lHVL+Z*VL+wgqOovZ31Qe1UW zATWIgIZ%?|;#c=Z?`wE?8VaKBF)t?Oh zkO$SvTpWz~F2r!VE9H3_NwXfvf>-Gz%|4(Xj>UVav-jXuFP%s{Op5i|yWNL>hI%5w z8(qVd4=@FK^r^Ms>Xj48UuxMxQIad(2%en|2jS}>Uts6H1NuALL{)5D#x5d)d}(VS zi6RqGR?+S%!AZWDJYWm9Wdv$IDT>e=Dxm5u0WCiSq5Ngs zPSYh77hNp5e9@w>rwbloaV%MahD5-&`8eu6G(iUWY@4!#Gzv9!{$L^h>8z}T1A8Cz zq`&PwpY+h*LEnJsrjqMMkg*by&XhH$PzVd#>lr1#-pU<3QBKazAZwNYo8h&$`9`Ac z6#B%4nUD92DJlPJ%5g3|0YcXGK%2O;3~ux2BRb z5}=O>#<Nd0%<-_?28VhqmIMx1J|k2;+a<+vS? zAJ*$o9&<<$!ek=wrQlHvUAIqZ*xymtP9TFc|Nt ze%{quW6Q;pQEflil#) z;~m#82vcy6^nyvTFwO<{O8l^M>!9w4^-yi0dKp>vUP!tD?m*r(SJO_W#W=#JKJ$`% z(WIgX+}pjl)!O5QnM+4b1X-dcD=OJo8JaN?JIup>QCA$uqB|>PbFm6a;aitDPqhNN z|4y4Lt#Nmr+|C`kb}r#wbz#6EUu#9fL?87g#zCbcfUiJV#l`fA=^Sgk6aoJGfc+Zo z_phei=EV=o*V_|*+q-qGb^#UlMi6#b5B~wU`+N)5Vs&_cm3mKG!UH(|hc~m{wA6;5 zblrndv%yPwN-RL&8;-vn+zB$4;Jok*+*$Qj?9KciKm4#RC30UdxLa$7`gyc5 zKp9m?!@WFUA|2tj-%ydma`OEl&o8UBHlnqrAh4NWsDcH+$7K%?uoBCF>}UvaRAzxY zfEu`81>x^(itc0qpET6Zg7D{4bzXsWUP~dO>^H6W0{SyYzKxi>akbb|HMv#mK>fJA z+b~lXm-TnTY2Uk?`nw?Pupa1wr*Q#BQNqCz3Z zBh%hmQ-71B*zlXNzr9#SAz-y_i$q?s8>8(6<1BbHb{ELM7eN2MgKL*NrpNu+gSFtE z3fwPuL(EX_7twG}fzCKX+_7O#Jlz(Rf7mnjMtkB@_#e|*2Yg8$rbdA6ck zOxO{+YQH%I?LPY-e$rzlA|zy4T_3r9pjU7sDNU z^(E!0egxvV6ZDRO8<{VF4Ft#cm%Y*kK*e!S|Y~sc|POLh6Jh zKPk;AUgm4NPBXg&;Ro_+pt>L+roSWkB}QR6Jd;klU~8(n{&Fj;7p(uktbI5pAio}0 z=aZP=&FS;vbWi_j2mVh$krZnMAbT#uv(CtW0r3A;oas&LYKpxKWKkn%K2n!Wr{TtH_^iT`n5&VTaQu3qN&EjrFbUFdIxw(_(MG=EE@+ubRTxN`JSU(TDO9Otql zVLAipftqiCU3w?VMWrE_D4*hOwb9MZOxAEaR${7j1K)W}k8?7+)tL;>oI~&pRCTUr zWl=cpKJR&>9Gn4xE%46CWSuczLQ_;pp&+yKlHKhdTuXlj$8e&QRF@xydoU6|%mX`N zD8@~-3%f~d)f7P$#C9H9PGm_M%P|D(V zI$)dSI`3Ey&;OITm*;=?&1Y*&G{lX)E#%0d4yeKychwH?j_ZDLgx)W}=k*Mv|1AM_ z7hD$*YbitTbXaS2J0?N%W~g`$A#0==^;~XbGyj`Nr5FqGFsd v@sN#tq`I`2tda7!I8g+dviH@Kuoh=n%*?>d49v{H%nZ!Tz|0IxgBkcgggp7H literal 15406 zcmeI22XGWs7{@n+UP2S3L^$aPNGO8f2q;P$!pLAj>4J!g1%@Vqg2ISIiVa3Zx(wjR z0D^#^v;jdWBO;(E2u={CNs&(C?|A`m)u@1aq`Xl^1b)H@_oDe_U(J`Vi++- ztWlzbfp!@K@)|}H!!TT~oB3KL4Pz#@gal7{4a3-4#xQD8hbFYRX-C;gYHF&>wlIVO z+E;~#L#S$7h4ww+1*l?MDM$fr=fIcH)~+gLF#*crQc_ZuQfD!`2s#d)gtVAQ-(lBY zcFM=lb{~2&`UE-^y&5$NibH+SBeWFCdVt#Og40TSWDR-?Jc|8QtHzYc$9hnE$ws9m zq%WZQqlcm&uxVd_^6{{nIyYbw=~L)Mf^?EqJ1ZM?%h0AZdJ0@8tr#|=s$12!v(csk z98`N`6O6R0YiFm8i$0E`cc4E-_k((N^{wpGuT5J$lXGyAd}phAvQd^_^`R4W2g5Yd zdVa6NDDtK4+U7L7<{1~X0?kjsgzIO>ne z;JYR|@mn_I$#sOg74z*_8#e)3y^xUL%(C==x?G(#^4bgVy)A6|^zS zyft+OQI^Ns7HXPy_tO4fc!RX$J*Y>18Kk%^{2{4ty2k5tbW!jA$!q;|pSLa4H0|z$ zx%8{CAU~%eNojwglHGMbFmA?bwrT2r~pNlzhN z815piZ*zX{zJsNyp9eG_Ekk?D1~S4O>tSGWih$u~Fq52mSKQEh26*~ZCK%Jn{^6YM`>>;VPk2d($B4%{Bh zuXj29R0j9Bqpgo%{{zEZZzHyy&>frBvoq0md-s=_G=0QEGtijQy7w~L6y0CFgKJUp zI~sjVh3I4$lo_9a+R$e)x^Iux*d7Cqz`Nvqo(1{q@HS-`y61s>Tb0p&H?Vvcu}>g| zAy)02wBhF&{OCFO2&VBEb~x$Ft}k7ue+=^PvAOKpIceuwH6GFg-$|f-UBJDmlQG~= zW6mw47lPyp9E8>IAw+CFh2t3cPypz4;QwA+q$r~I1D{1;0UQAHyGb%-1-`2}alh%O z0Mvy<`qdigefSytS1E|122AoeD(9P}(Q zwKt7M+@?=rj)C4hi@oUkbLch1dcL1Wh`+UYZ8=F3cX6&k>$=mVJ3xl^*xBJ)iMb8& zAA^(7OV@zb?+dqPhxmfUK)l+c_9b44&Q7&HYl+qxG*f4G!F)!doS0`2ulB9Ur0aq9 zm5a!ikAy#``y@`SkGG+<$Eb?Ff|jg>I!^m@x<6vnoT_)>xZ*|YotEglX*v{f(kJ~E zgjk3B)c#LrV>&z58IksZmti=`E`fYe9;0$}NHI~bI#deOXJ*QXRqKmd=>2HLuenL* zfP295H$tjar))4<=e1QlxUa`92I`lGfp+EL*y*D9LKOeLw*EkyN-7opF*mX{% zbF-`H!{`L`FmxyHy8da(3sP?#x`2OKxOw_4LEjre=Y9IUR(oh&%RbQaDF@w1{|r87 zPh?eBzj5n~ehS2sw|o}ZzfwQp#_Qi6pT%G$tbzex#`q7($6yIu!?%@a-rWFgKxYU# zSLg!i)`{3=z#7QZZ#@2Sn|b;xOnmdvozaKUrf3Z5p6Z-Gkf@&CpN8l!;2g{XpYI|0 zSKu$|)ir%Pq3KU!^I6dOo4@G2cn)-*0}Abbq(UNWC#`3=1)76<%%gi^zJ%`QMBHW{ z^rzoGYS8yaw2yfFY3v{HDX`AyIjp2e&vC1&pf#V3gYsl`Eqd=~p^KXS!_oAU4|;(9 zCq~aUonT)9IyVa{)Voe4J+sE7cY#Fjj%@dU{Ctb95@y_a=|ksvJJHM0lhBr9#N%Jz zYWM#VQ7;tH*wJ&-cSP$%eHYKeFz7{DSpWY(ADZ*Gp>@A>PkulL60P@^f;l$@s;hCM zYkdM5f`0dS5u45*^T;=>y-ITU)~f(EO*dqjzy6XpB5h zjJp43%$U+d--3y-2TPXzKa1715`*?14M6{U*@(FILa=v!8g(=#E&tcj%1>6pq=GWoWPq(IaI;M>FM_d&_d!KaIfGL@nA6FT&gmhH@Z`1DDh)e7K-Dn@drv03L z!_?a99dx!m<0HPN#OJeyBCluC2j;<4vM+n`9nqT0DnYjSeTZ1JhtM7-O~`9cq;G*l z=Y5wz`?C{pj57Vdna@7W?m1_Lop{>8A#e+^X$^7Kll4)QNwlW5Jg3Quc>R44pw9I= z64OynusT!K?>g=)w*8W|EKitKhk+((O$G1=)KkW&|X+^x`njf2|XXJL*gPG zkJ&%Iv|lI$rJw@&YM|fK+LG5C|1oKO&yIt}`6?I$`?1ePOEl&ekawB=Mncnne*DCv z8-dQV^c%rw^mJGTTj2;?z-}j3DVPIVn`+;+lyoAb?PnqpyZXe3_5@l>mxbF%YmHnJ nYG)9ApS9o8ddZRVhDcp=Zoj#AAomXB-htdZkb4Khy#xONQWqeZ From 1f249a33386a44e6735450c3326ef7fb01879ad0 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Thu, 17 Feb 2022 19:29:41 +0100 Subject: [PATCH 10/12] iox-#743 Adapt output dir for .svg's Signed-off-by: Simon Hoinkis --- tools/scripts/generate_plantuml_svg.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/scripts/generate_plantuml_svg.sh b/tools/scripts/generate_plantuml_svg.sh index c4babc952e..c2bad31d30 100755 --- a/tools/scripts/generate_plantuml_svg.sh +++ b/tools/scripts/generate_plantuml_svg.sh @@ -23,7 +23,8 @@ set -e WORKSPACE="$(git rev-parse --show-toplevel)" PUML_DIR="$WORKSPACE/doc/design/diagrams" -EXPORT_DIR=${1:-$PUML_DIR} +IMPORT_DIR=${1:-$PUML_DIR} +EXPORT_DIR="$WORKSPACE/doc/website/images/" TEMP_DIR="/var/tmp/iceoryx" # this is persistent across reboots PLANTUML_DIR="$TEMP_DIR/plantuml-jar-mit-1.2021.5" NUM_THREADS=1 @@ -59,6 +60,7 @@ java -jar $PLANTUML_DIR/plantuml.jar \ -nometadata \ -nbthread "$NUM_THREADS" \ -tsvg \ - -I "$EXPORT_DIR/**.puml" + -I "$IMPORT_DIR/**.puml" \ + -output "$EXPORT_DIR" echo " [i] Finished" From 3fc48d62500cbfec3aa315f44d078451cd50da84 Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Thu, 17 Feb 2022 20:06:46 +0100 Subject: [PATCH 11/12] iox-#743 Add todo in design article Signed-off-by: Simon Hoinkis --- doc/website/design/design_article.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 doc/website/design/design_article.md diff --git a/doc/website/design/design_article.md b/doc/website/design/design_article.md new file mode 100644 index 0000000000..3fd318539d --- /dev/null +++ b/doc/website/design/design_article.md @@ -0,0 +1,3 @@ +# Design article + + From 01ccc13edd39dc8067af7e0fa95dbf4641bb711b Mon Sep 17 00:00:00 2001 From: Simon Hoinkis Date: Fri, 18 Feb 2022 09:38:23 +0100 Subject: [PATCH 12/12] iox-#743 Update iceoryx users in docs Signed-off-by: Simon Hoinkis --- doc/design/request_response_communication.md | 16 ++++++++-------- doc/design/service-discovery.md | 10 +++++----- doc/website/getting-started/what-is-iceoryx.md | 2 +- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/doc/design/request_response_communication.md b/doc/design/request_response_communication.md index e268a5da98..05212db24f 100644 --- a/doc/design/request_response_communication.md +++ b/doc/design/request_response_communication.md @@ -46,20 +46,20 @@ The `Client` and `Server` are reusing the `ChunkSender` and `ChunkReceiver` buil #### Typed API -![typed API](typed_api.svg) +![typed API](../website/images/typed_api.svg) Since the `Response` is tied to a specific `Request`, the `loan` method takes a `Request` to populate the `Response` with the correct settings. Alternatively, a `Request` could have a `createResponse` method which returns a `Response` with the correct settings. #### Untyped API -![untyped API](untyped_api.svg) +![untyped API](../website/images/untyped_api.svg) Similar to the the typed API, `loan` takes a pointer to a `RequestHeader` to populate the `ResponseHeader` with the correct settings. #### Client Port -![client port](client_port.svg) +![client port](../website/images/client_port.svg) The `ClientPortData` is located in the shared memory and contain only the data but no methods to access them. `ClientPortUser` is the class providing the methods for the user access and `ClientPortRouDi` provides the @@ -67,7 +67,7 @@ interface RouDi needs to connect the client to the server and to cleanup the por #### Server Port -![server port](server_port.svg) +![server port](../website/images/server_port.svg) Similar to the Client Port, the Server Port has `ServerPortData` which is located in the shared memory and contain only the data but no methods to access them. `ServerPortUser` is the class providing the methods for the user access and `ServerPortRouDi` provides the @@ -77,7 +77,7 @@ It must be ensured that only one server with a given `ServiceDescription` can ru #### Request/Response Header -![rpc header](request_response_header.svg) +![rpc header](../website/images/request_response_header.svg) Since request and response need to encode different meta-information, we also need different header for the messages. The common data is aggregated in `RpcBaseHeader` which contains a `cxx::UniqueId` to the `ClientChunkQueueData_t` and a sequence ID. @@ -101,7 +101,7 @@ The `RequestHeader` has also the option to specify a message as fire and forget, #### Client/Server Options -![client and server options](client_and_server_options.svg) +![client and server options](../website/images/client_and_server_options.svg) The client and server options can be used to control certain aspects of the clients and servers. Beside setting the capacity of the queues and defining whether a client should be connected and a server offering on creation, @@ -114,14 +114,14 @@ If the options don't match, the client will not be connected to the server, simi The client is guided by the following state machine. -![client state machine](client_state_machine.svg) +![client state machine](../website/images/client_state_machine.svg) Similar to the subscriber state machine, the client passes it's response queue with the `CONNECT` CaPro message to the server. The server will pass its request queue with the `ACK` CaPro message to the client. Following is a sequence diagram which shows all this cases -![client and server service discovery](client_and_server_service_discovery.svg) +![client and server service discovery](../website/images/client_and_server_service_discovery.svg) ### Code example diff --git a/doc/design/service-discovery.md b/doc/design/service-discovery.md index 905d27d7f3..9dbc761a15 100644 --- a/doc/design/service-discovery.md +++ b/doc/design/service-discovery.md @@ -128,9 +128,9 @@ Contra: * Built-in topic approach based on `InterfacePort`'s * Sniff and intercept `CaproMessage` -![overview diagram](overview-alternative-b.svg) +![overview diagram](../website/images/overview-alternative-b.svg) -![sequence diagram](sequence-diagram-alternative-b.svg) +![sequence diagram](../website/images/sequence-diagram-alternative-b.svg) Pro: @@ -172,9 +172,9 @@ Create a new publisher in RouDi which sends a `ServiceRegistryTopic`. This publi change in the service registry and to transmit the service discovery registry. The complete old service registry (saved locally) would be compared to the new service registry in a new class, extending the public user API. -![overview diagram](overview-alternative-d.svg) +![overview diagram](../website/images/overview-alternative-d.svg) -![sequence diagram](sequence-diagram-alternative-d.svg) +![sequence diagram](../website/images/sequence-diagram-alternative-d.svg) Pro: @@ -416,5 +416,5 @@ PoshRuntime::findService(const capro::ServiceDescription& serviceDescription) no * [x] How does the SOME/IP-SD service discovery API look like? * [x] What does a `ros topic list` do in rmw_iceoryx? * [ ] Filter for `ServiceDescription::EventString` needed by AUTOSAR? Not supported by `ServiceRegistry::find()` as of today -* [ ] Decision on mapping table between iceory and DDS (see [Overview](overview.md)) +* [ ] Decision on mapping table between iceory and DDS (see [Overview](../website/getting-started/overview.md)) * [ ] Current mapping table between iceoryx and DDS does not work with service discovery diff --git a/doc/website/getting-started/what-is-iceoryx.md b/doc/website/getting-started/what-is-iceoryx.md index 6cc49dc430..bf87d97a7b 100644 --- a/doc/website/getting-started/what-is-iceoryx.md +++ b/doc/website/getting-started/what-is-iceoryx.md @@ -41,7 +41,7 @@ That iceoryx has the right set of features can be seen from the already existing frameworks such as [Eclipse Cyclone DDS](https://github.com/eclipse-cyclonedds/cyclonedds), [eCAL from Continental](https://continental.github.io/ecal/), [RTA-VRTE from ETAS](https://www.etas.com/en/products/rta-vrte.php) and - [Apex.OS from Apex.AI](https://www.apex.ai/apex-os). + [Apex.Middleware from Apex.AI](https://www.apex.ai/apex-middleware). ## Dependable