Release v4.1.0 - Quality of Life Improvements

.github/workflows/build.yml

@@ -8,7 +8,7 @@ on:
 
 jobs:
   build:
-    name: Build & Test
+    name: Build, Test, Report
     runs-on: windows-latest
 
     steps:
@@ -23,7 +23,16 @@ jobs:
       run: dotnet restore RabbitMQ.Dataflows.sln
 
     - name: Build
-      run: dotnet build RabbitMQ.Dataflows.sln --configuration Release
+      run: dotnet build RabbitMQ.Dataflows.sln --configuration Release --no-restore
 
     - name: Test
-      run: dotnet test "./tests/UnitTests/UnitTests.csproj"
+      run: dotnet test "./tests/UnitTests/UnitTests.csproj" --configuration Release --no-build --verbosity normal
+           /p:CollectCoverage=true
+           /p:CoverletOutput='../../coverage.cobertura.xml'
+           /p:CoverletOutputFormat=cobertura
+
+    - name: Upload Coverage Report
+      uses: codacy/codacy-coverage-reporter-action@v1
+      with:
+        project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
+        coverage-reports: 'coverage.cobertura.xml'

.github/workflows/publish.yml

@@ -79,15 +79,6 @@ jobs:
           PROJECT_FILE_PATH: src/HouseofCat.Hashing/HouseofCat.Hashing.csproj
           INCLUDE_SYMBOLS: true
 
-    - name: Publish HouseofCat.Metrics
-      uses: alirezanet/publish-nuget@v3.1.0
-      continue-on-error: true
-      with:
-          NUGET_KEY: ${{secrets.NUGET_API_KEY}}
-          VERSION_FILE_PATH: version.props
-          PROJECT_FILE_PATH: src/HouseofCat.Metrics/HouseofCat.Metrics.csproj
-          INCLUDE_SYMBOLS: true
-
     - name: Publish HouseofCat.RabbitMQ
       uses: alirezanet/publish-nuget@v3.1.0
       continue-on-error: true

.gitignore

@@ -348,3 +348,4 @@ MigrationBackup/
 
 # Ionide (cross platform F# VS Code tools) working folder
 .ionide/
+tests/UnitTests/coverage.cobertura.xml

README.md

@@ -3,87 +3,76 @@
 A RabbitMQ library of `.NET` tools to help rapidly develop well performant services or
 to just help manage durable connectivity with the `RabbitMQ.Client`!  
 
+## Status
+
+Test Server: `Windows 11`  
+RabbitMQ Server: `v3.13`  
+Erlang: `v26.2.3`  
+
+![Release](https://img.shields.io/github/v/release/houseofcat/RabbitMQ.Dataflows)
+[![build](https://github.com/houseofcat/RabbitMQ.Dataflows/workflows/build/badge.svg)](https://github.com/houseofcat/RabbitMQ.Dataflows/actions/workflows/build.yml)  
+[![Codacy Badge1](https://app.codacy.com/project/badge/Grade/2ac2a6f51a8c4efd88135bcb835e3a97)](https://app.codacy.com/gh/houseofcat/RabbitMQ.Dataflows/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)   
+[![Codacy Badge2](https://app.codacy.com/project/badge/Coverage/2ac2a6f51a8c4efd88135bcb835e3a97)](https://app.codacy.com/gh/houseofcat/RabbitMQ.Dataflows/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_coverage)
+
 ## Why Make A RabbitMQ Powered Dataflow?  
 
 `Dataflows` have configurable concurrency, serialization, monitoring, compression, and
 encryption all as first class citizens. This paradigm allows developers to just focus on
-the important stuff - getting work done
+the important stuff - getting work done!
+
+Here are some features ready with `RabbitMQ.Dataflows`.
 
-Here are some features ready with RabbitMQ and Dataflows today!
+### Workflows
 
-### Queueing
-* Async Processing, batch processing, consumer cloning and more!    
-* Queue-based Retriability via Ack/Nack.
-* Async Error Handling (simplify functional error handling by allowing it to throw.)
-* A RabbitMQ ConnectionPool and ChannelPool (connection durability) provided by
-`namespace HouseofCat.RabbitMQ.Pools;`.  
+ * Pipelines (v1) are light weight workflow-esque function-as-step executor.
+   * All steps process in the order provided allowing you to still control order of execution.  
+   * Configurable parallelism and concurrency built-in.
+   * No automatic OpenTelemetry.
+   * No automatic exception catching/async error handling (thrown exceptions kill Pipelines).
+ * Dataflows (v2) are the more complex, yet complete, workflow class.
+   * All steps process in the order provided allowing you to still control order of execution.  
+   * Configurable parallelism and concurrency built-in.
+   * Async processing, batch processing, consumer cloning and more!  
+   * Async error handling (simplify functional error handling by allowing functions to throw).
+   * AutoPublish (to the next Queue) functionality built-in.
+   * OpenTelemetry with native distributed tracing for Publish/Consumer. 
 
 ### Built-Ins
-* Supports `ILogger<T>` via LogHelpers. 
-* Configurable concurrency/parallelism, no code changes required.  
+
+* A RabbitMQ ConnectionPool and ChannelPool (connection durability).  
+* Supports `ILogger<T>` via LogHelpers static singleton. 
 * Contracted `IWorkState` simplifies functional generic returns and integration.  
-* Has `Json` (System.Text.Json and Newtonsoft) and `MessagePack` serialization providers.
-* Allows seamless encryption/decryption steps.  
-* Allows seamless compression/decompression steps.  
-* Async Error Handling with Predicate triggers and an actionable callback.  
-
-### Core Interchangeability
-* Allows you to replace serialization provider with `ISerializationProvider` and have basic 
-implementations.  
-* Allows you to replace encryption provider with `IEncryptionProvider` and have basic 
-implementations.  
-* Allows you to replace compression provider with `ICompressionProvider` and have basic 
-implementations.   
-
-### Business Logic
-* All steps process in the order provided allowing you to still control order of execution.  
-* All automatically subscribed to Async Error handling by `WorkState.IsFaulted` flag.  
+* Has `ISerializationProvider` support for `System.Text.Json` and `MessagePack`.
+  * Easy to write your own providers. 
+* Has `ICompressionProvider` built-in support for Gzip, Deflate, Brotli, and LZ4.
+* Has `IEncryptionProvider` built-in support for AesGcm and BouncyCastle AesGcm.
+* RecyclableMemoryStream options for `ICompressionProvider` and `IEncryptionProvider`.
+* Publishers AutoPublish seamless encryption/decryption steps (adds headers too).  
+* Consumers allow seamless compression/decompression steps (uses headers).  
+* Publisher/Consumers have built-in `OpenTelemetry` support.
 
 ### Testing
+
 * All built-in steps will have integration tests that should remove concerns from end-user 
 developer.   
 * Future case will include much more complex abstract UnitTesting as time allows.  
 * The developer should only need to unit test their functional business code.  
 
-## Implicit Benefits
-
-The benefits of a dataflow pattern extend beyond fancy machine learning and Tensorflows or
-high throughput GCP Dataflow for mass computation. When brought to the service level, it
-helps organize your code into more manageable blocks. You can still write monolithic
-functions, but you would be hamstringing yourself and scarificing concurrency and
-parallelism. By designing code into small functional steps, you always write better,
-cleaner, code reduced with cyclomatic complexity. That very same code is easier to
-UnitTest. The orchestration of the function calls are the order they are added allowing
-you extend the original functionality infinitely. You don't have to write deserialization
-or post-processing encryption/compression as they all baked in. Designing from the ground
-up with concurrency and parallelism, you stay nimble and fast - able to scale up internally,
-before horizontally and vertically, saving costs. All without needing code changed or
-refactored.
-
-Lastly, after everything is said and done, all your business code is re-usable. Should
-you decide to abandon this workflow (:worried:) for a different mechanism, engine, or
-what not, all of your code will happily port to whatever other project / flow you are
-working with and so will all your testing making it a win win.  
-
 ## Help & Guides
- * Getting started with a [RabbitMQ.Dataflow.ConnectionPool](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/ConnectionPools.md).
- * Getting started with a [RabbitMQ.Dataflow.ChannelPool](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/ChannelPools.md).
- * Getting started with a [ChannelPool and IModel.BasicPublish](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/BasicPublish.md).
- * Getting started with a [ChannelPool and IModel.BasicGet](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/BasicGet.md).
- * Getting started with a [ChannelPool and IModel.BasicConsume](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/BasicConsume.md).
- * Getting started with RabbitMQ.Dataflow [Serialization](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/Serialization.md).
-
-You can also find various library examples inside the `tests/UnitTests` or the `tests/RabbitMQ.Console.Test` project.
 
+ * Getting started with *RabbitMQ.Dataflows* [ConnectionPool](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/ConnectionPools.md).
+ * Getting started with *RabbitMQ.Dataflows* [ChannelPool](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/ChannelPools.md).
+ * Getting started with *RabbitMQ.Dataflows* [ChannelPool and BasicPublish](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/BasicPublish.md).
+ * Getting started with *RabbitMQ.Dataflows* [ChannelPool and BasicGet](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/BasicGet.md).
+ * Getting started with *RabbitMQ.Dataflows* [ChannelPool and BasicConsume](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/BasicConsume.md).
+ * Getting started with *RabbitMQ.Dataflows* [Serialization](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/Serialization.md).  
+ * Getting started with *RabbitMQ.Dataflows* [Publisher](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/Publisher.md).  
+ * Getting started with *RabbitMQ.Dataflows* [AutoPublisher](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/AutoPublisher.md).   
+ * Getting started with *RabbitMQ.Dataflows* [Consumer](https://github.com/houseofcat/RabbitMQ.Dataflows/blob/main/guides/rabbitmq/Consumer.md).  
 
-## Status
+More to come!
 
-Test Server: `Windows 11`  
-RabbitMQ Server: `v3.13`  
-Erlang: `v26.2.3`  
-
-[![Codacy Badge](https://api.codacy.com/project/badge/Grade/9dbb20a30ada48caae4b92a83628f45e)](https://app.codacy.com/gh/houseofcat/RabbitMQ.Dataflows/dashboard)  
-[![build](https://github.com/houseofcat/RabbitMQ.Dataflows/workflows/build/badge.svg)](https://github.com/houseofcat/RabbitMQ.Dataflows/actions/workflows/build.yml)
+You can also find various library examples inside the `tests/UnitTests` or the `tests/RabbitMQ.Console.Test` project.
 
 # Main RabbitMQ Library
 
@@ -146,14 +135,6 @@ well as the interface to make your own.
  * Supports BouncyCastle AesGcm 128/192/256.  
  * Supports RecyclableMemoryStream variants.  
 
-## HouseofCat.Metrics
-[![NuGet](https://img.shields.io/nuget/v/HouseofCat.Metrics.svg)](https://www.nuget.org/packages/HouseofCat.Metrics/)  
-[![NuGet](https://img.shields.io/nuget/dt/HouseofCat.Metrics.svg)](https://www.nuget.org/packages/HouseofCat.Metrics/)  
-
-A library that has a collection of .NET IMetricsProvider or the interface to make your own.
-
- * Supports Prometheus.Net.  
-
 # Non-Critical Library Integrations
 
 ## HouseofCat.Data

RabbitMQ.Dataflows.sln

@@ -81,6 +81,8 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "rabbitmq", "rabbitmq", "{5C
 		guides\rabbitmq\BasicPublish.md = guides\rabbitmq\BasicPublish.md
 		guides\rabbitmq\ChannelPools.md = guides\rabbitmq\ChannelPools.md
 		guides\rabbitmq\ConnectionPools.md = guides\rabbitmq\ConnectionPools.md
+		guides\rabbitmq\Consumer.md = guides\rabbitmq\Consumer.md
+		guides\rabbitmq\Publisher.md = guides\rabbitmq\Publisher.md
 		guides\rabbitmq\Serialization.md = guides\rabbitmq\Serialization.md
 	EndProjectSection
 EndProject

guides/csharp/TopLevel.md

@@ -154,8 +154,8 @@ Let me copy in a basic HoC config with our consumer settings in it. This file ne
       "Exclusive": false,
       "QueueName": "TestQueue",
       "QueueArguments": null,
-      "TargetQueueName": "TestTargetQueue",
-      "TargetQueueArgs": null,
+      "SendQueueName": "TestTargetQueue",
+      "SendQueueArgs": null,
       "ErrorQueueName": "TestQueue.Error",
       "ErrorQueueArgs": null,
       "BuildQueues": true,
@@ -167,7 +167,9 @@ Let me copy in a basic HoC config with our consumer settings in it. This file ne
       "WorkflowConsumerCount": 1,
       "WorkflowBatchSize": 5,
       "WorkflowEnsureOrdered": false,
-      "WorkflowWaitForCompletion": false
+      "WorkflowWaitForCompletion": false,
+      "WorkflowSendCompressed": false,
+      "WorkflowSendEncrypted": false
     }
   }
 }
@@ -193,6 +195,8 @@ var rabbitService = new RabbitService(
     compressionProvider: null,
     loggerFactory);
 
+await rabbitService.StartAsync();
+
 var consumer = rabbitService.GetConsumer("HoC-Consumer");
 await consumer.StartConsumerAsync();
 ```
@@ -250,6 +254,7 @@ var rabbitService = new RabbitService(
     compressionProvider: null,
     loggerFactory);
 
+await rabbitService.StartAsync();
 var consumer = rabbitService.GetConsumer("HoC-Consumer");
 await consumer.StartConsumerAsync();