From 36b1183320d643f6efc039972164ab549ae26628 Mon Sep 17 00:00:00 2001
From: vboxuser <vboxuser@Ubuntu.myguest.virtualbox.org>
Date: Fri, 3 Feb 2023 11:31:56 -0800
Subject: [PATCH] documenation for metaserver/mqtt local

Documentation for using metaserver will allow users to run examples with a local broker.
It also allows for mqtt local brokers.
This decreases the chances of any job ID collisions.

Modifications to the config.json for the mnist example were made in order to make it easier to switch to a local broker.
The readme does indicate how to do this for other examples now.
---
 docs/08-flame-sdk.md                          | 79 ++++++++++++++++++-
 .../examples/mnist/aggregator/config.json     |  8 +-
 .../flame/examples/mnist/trainer/config.json  |  8 +-
 3 files changed, 88 insertions(+), 7 deletions(-)

diff --git a/docs/08-flame-sdk.md b/docs/08-flame-sdk.md
index 88bfc5c0c..a4f6436c9 100644
--- a/docs/08-flame-sdk.md
+++ b/docs/08-flame-sdk.md
@@ -20,17 +20,90 @@ make install
 ## Quickstart
 
 ### Configuring Brokers
-As the flame system uses MQTT brokers to exchange messages during federated learning, to run the python library locally, you could either 1) install a local MQTT broker 2) use a public MQTT broker. Here we'll illustrate the second option.
 
-Go to any examples that you wish to run locally in `examples` directory, change the `host` from `"flame-mosquitto"` to `broker.hivemq.com` in the `config.json` files of both the trainer and aggregator.
+The following brokers are all for local testing.
+If you wish to run federated learning accross multiple machines, please consider using the MQTT public broker.
+This means setting `backend` and the `mqtt` broker in the config file as follows:
+
+```json
+    "backend": "mqtt",
+    "brokers": [
+        {
+            "host": "broker.hivemq.com",
+            "sort": "mqtt"
+        }
+    ]
+```
+
+However, this may lead to job ID collisions since it is a public broker.
+Thus, for local testing, we recommend using either of the two options below.
+
+#### Local MQTT Broker
+
+Since the flame system uses MQTT brokers to exchange messages during federated learning, to run the python library locally, you may install a local MQTT broker as shown below.
+
+```bash
+sudo apt update
+sudo apt install -y mosquitto
+sudo systemctl status mosquitto
+```
+
+The last command should display something similar to this:
+
+```bash
+mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
+     Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor pre>
+     Active: active (running) since Fri 2023-02-03 14:05:55 PST; 1h 20min ago
+       Docs: man:mosquitto.conf(5)
+             man:mosquitto(8)
+   Main PID: 75525 (mosquitto)
+      Tasks: 3 (limit: 9449)
+     Memory: 1.9M
+     CGroup: /system.slice/mosquitto.service
+             └─75525 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
+```
+
+That confirms that the mosquitto service is active.
+From now on, you may use `sudo systemctl stop mosquitto` to stop the mosquitto service, `sudo systemctl start mosquitto` to start the service, and `sudo systemctl restart mosquitto` to restart the service.
+
+Go ahead and change the two config files in `mnist/trainer` and `mnist/aggregator` to make sure `backend` is `mqtt`.
+
+```json
+    "backend": "mqtt",
+    "brokers": [
+        {
+            "host": "localhost",
+            "sort": "mqtt"
+        },
+	{
+	    "host": "localhost:10104",
+	    "sort": "p2p"
+	}
+    ]
+```
+
+Note that if you also want to use the local `mqtt` broker for other examples you should make sure that the `mqtt` broker has `host` set to `localhost`.
+
+#### P2P
+
+To start a `p2p` broker, go to the top `/flame` directory and run:
+
+```bash
+make install
+cd ~
+sudo ./.flame/bin/metaserver
+```
+
+After changing the two config files in `mnist/trainer` and `mnist/aggregator` so that `backend` is set to `p2p`, continue to the next section.
 
 ### Running an Example
 
 In order to run this example, you will need to open two terminals.
 
-In the first terminal, run the following command:
+In the first terminal, run the following commands:
 
 ```bash
+conda activate flame
 cd examples/mnist/trainer
 
 python keras/main.py config.json
diff --git a/lib/python/flame/examples/mnist/aggregator/config.json b/lib/python/flame/examples/mnist/aggregator/config.json
index 8ffb6ea74..24d81e9e6 100644
--- a/lib/python/flame/examples/mnist/aggregator/config.json
+++ b/lib/python/flame/examples/mnist/aggregator/config.json
@@ -3,9 +3,13 @@
     "backend": "mqtt",
     "brokers": [
         {
-            "host": "broker.hivemq.com",
+            "host": "localhost",
             "sort": "mqtt"
-        }
+        },
+	{
+	    "host": "localhost:10104",
+	    "sort": "p2p"
+	}
     ],
     "channels": [
         {
diff --git a/lib/python/flame/examples/mnist/trainer/config.json b/lib/python/flame/examples/mnist/trainer/config.json
index c689269a8..cf049e888 100644
--- a/lib/python/flame/examples/mnist/trainer/config.json
+++ b/lib/python/flame/examples/mnist/trainer/config.json
@@ -3,9 +3,13 @@
     "backend": "mqtt",
     "brokers": [
         {
-            "host": "broker.hivemq.com",
+            "host": "localhost",
             "sort": "mqtt"
-        }
+        },
+	{
+	    "host": "localhost:10104",
+	    "sort": "p2p"
+	}
     ],
     "channels": [
         {