Docs

Signed-off-by: Tomasz Maruszak <maruszaktomasz@gmail.com>

docs/intro.md

@@ -2,6 +2,12 @@
 
 - [About](#about)
 - [Configuration](#configuration)
+  - [Transport](#transport)
+    - [IP / UDP Transport](#ip--udp-transport)
+  - [Serialization](#serialization)
+    - [JSON Serialization](#json-serialization)
+  - [Cluster Persistence](#cluster-persistence)
+    - [Local File Persistence](#local-file-persistence)
 - [Cluster Membership](#cluster-membership)
   - [SWIM Membership](#swim-membership)
   - [Static Membership](#static-membership)
@@ -11,10 +17,6 @@
     - [State Machine](#state-machine)
     - [Configuration Parameters](#configuration-parameters)
   - [Leader Request Delegation ASP.NET Core Middleware](#leader-request-delegation-aspnet-core-middleware)
-- [Cluster Persistence](#cluster-persistence)
-- [Serialization](#serialization)
-- [Transport](#transport)
-  - [IP / UDP Transport](#ip--udp-transport)
 
 # About
 
@@ -27,7 +29,125 @@ Think about building the next distributed stream processing system (Flink), data
 
 # Configuration
 
-ToDo
+The SlimCluster configuration consists of adding it to the MSDI container using the `services.AddSlimCluster(cfg => {})` method:
+
+```cs
+// IServiceCollection services;
+
+services.AddSlimCluster(cfg =>
+{
+    cfg.ClusterId = "MyCluster";
+    // This will use the machine name, in Kubernetes this will be the pod name
+    cfg.NodeId = Environment.MachineName;
+
+    // Plugin: Transport will be over UDP/IP
+    cfg.AddIpTransport(opts =>
+    {
+        opts.Port = 60001; // Any available port can be used, this value is default
+        opts.MulticastGroupAddress = "239.1.1.1" // Any valid multicast group can be used, this value is default
+    });
+
+    // Plugin: Protocol messages (and logs/commands) will be serialized using JSON
+    cfg.AddJsonSerialization();
+
+    // Plugin: Cluster state will saved into the local json file in between node restarts
+    cfg.AddPersistenceUsingLocalFile("cluster-state.json");
+
+    // Other plugins: membership, consensus, tools
+});
+```
+
+As part of the configuration we can set:
+
+- What is the ClusterId (logical identifier) and name of the Node.
+- What transport will be used (in the above example IP protocol).
+- What serialization will be used for the protocol messages (membership, consensus, etc).
+- How should the cluster state be persisted in between node instance restarts (this is optional).
+- What membership algorithm to use (SWIM).
+- What consensus algorithm to use (Raft).
+- What other plugins to add.
+
+SlimCluster registers as an `IHostedService` that works with MS Hosting abstraction.
+This ensures the cluster starts when any [NET Generic Host](https://learn.microsoft.com/en-us/dotnet/core/extensions/generic-host) compatible runtime starts (ASP.NET Core application).
+
+## Transport
+
+Package: [SlimCluster.Transport](https://www.nuget.org/packages/SlimCluster.Transport)
+
+The transport allows the Nodes in the cluster to exchange protocol messages.
+
+Currently the only possible transport option is IP protocol, where the inter-Node communication happens using UDP sockets.
+
+### IP / UDP Transport
+
+Package: [SlimCluster.Transport.Ip](https://www.nuget.org/packages/SlimCluster.Transport.Ip)
+
+Configures SlimCluster to use the IP/UDP protocol for inter-Node communication.
+
+```cs
+cfg.AddIpTransport(opts =>
+{
+    opts.Port = builder.Configuration.GetValue<int>("UdpPort");
+    opts.MulticastGroupAddress = builder.Configuration.GetValue<string>("UdpMulticastGroupAddress")!;
+});
+```
+
+By design the various plugins that are added to SlimCluster share the same IP port.
+The messages are multiplexed into the same Node endpoint, and then routed to the relevant plugin/layer that is able to handle the messages.
+Thanks to this SC requires only one port.
+In this scenario consensus (Raft) and membership (SWIM) protocol messages go through the same UDP port.
+
+![](images/SlimCluster.jpg)
+
+The `MulticastGroupAddress` property allows to set the multicast group used in the initial member discovery.
+In the future there will be also option to avoid multicast groups.
+
+## Serialization
+
+Package: [SlimCluster.Serialization](https://www.nuget.org/packages/SlimCluster.Serialization)
+
+Serialization is needed for protocol messages for inter-Node communication.
+
+Currently the only possible serialization option is JSON.
+In the near future there are plans to add [ProtoBuf](https://github.com/protocolbuffers/protobuf) binary serialization for performance and payload size optimization.
+
+To write a custom serializer, simply provide and implementation for [ISerializer](../src/SlimCluster.Serialization/ISerializer.cs) in the MSDI container.
+
+### JSON Serialization
+
+Package: [SlimCluster.Serialization.Json](https://www.nuget.org/packages/SlimCluster.Serialization.Json)
+
+```cs
+// Plugin: Protocol messages (and logs/commands) will be serialized using JSON
+cfg.AddJsonSerialization();
+```
+
+## Cluster Persistence
+
+Package: [SlimCluster.Persistence](https://www.nuget.org/packages/SlimCluster.Persistence)
+
+The Node (service instance) needs to persist the minimal cluster state in between instance restarts (or crushes) to quickly catch up with the other Nodes in the cluster.
+Depending on the plugins and configuration used, such state might include:
+
+- Last known member list
+- Last known leader
+- Algorithm state (Raft, SWIM)
+
+While this is not required, it allows the restarted (or crushed) Node to faster catch up with the other members in the Cluster and converge into the correct cluster state.
+
+Currently the only possible option is LocalFile which stores the state in an local JSON file.
+In the near future there are plans to add some connectors to Cloud Provider storage offerings (Azure, AWS, GCP).
+
+To write a custom state persitence strategy, simply provide and implementation for [IClusterPersistenceService](../src/SlimCluster.Persistence/IClusterPersistenceService.cs) in the MSDI container.
+
+### Local File Persistence
+
+Package: [SlimCluster.Persistence.LocalFile](https://www.nuget.org/packages/SlimCluster.Persistence.LocalFile)
+
+```cs
+// Plugin: Cluster state will saved into the local json file in between node restarts
+cfg.AddPersistenceUsingLocalFile("cluster-state.json");
+```
 
 # Cluster Membership
 
@@ -105,19 +225,3 @@ ToDo
 ## Leader Request Delegation ASP.NET Core Middleware
 
 ToDo
-
-# Cluster Persistence
-
-ToDo
-
-# Serialization
-
-ToDo
-
-# Transport
-
-ToDo
-
-## IP / UDP Transport
-
-ToDo

src/Samples/SlimCluster.Samples.Service/Program.cs

@@ -33,6 +33,12 @@
         opts.MulticastGroupAddress = builder.Configuration.GetValue<string>("UdpMulticastGroupAddress")!;
     });
 
+    // Protocol messages (and logs/commands) will be serialized using JSON
+    cfg.AddJsonSerialization();
+
+    // Cluster state will saved into the local json file in between node restarts
+    cfg.AddPersistenceUsingLocalFile("cluster-state.json");
+
     // Setup Swim Cluster Membership
     cfg.AddSwimMembership(opts =>
     {
@@ -53,12 +59,6 @@
         // opts.LogSerializerType = typeof(JsonSerializer);
     });
 
-    // Protocol messages (and logs/commands) will be serialized using JSON
-    cfg.AddJsonSerialization();
-
-    // Cluster state will saved into the local json file in between node restarts
-    cfg.AddPersistenceUsingLocalFile("cluster-state.json");
-
     cfg.AddAspNetCore(opts =>
     {
         // Route all ASP.NET API requests for the Counter endpoint to the Leader node for handling