Skip to content

Commit

Permalink
documenation for metaserver/mqtt local (#322)
Browse files Browse the repository at this point in the history
Documentation for using metaserver will allow users to run examples with a local broker.
It also allows for mqtt local brokers.
This decreases the chances of any job ID collisions.

Modifications to the config.json for the mnist example were made in order to make it easier to switch to a local broker.
The readme does indicate how to do this for other examples now.

Co-authored-by: vboxuser <vboxuser@Ubuntu.myguest.virtualbox.org>
  • Loading branch information
GustavBaumgart and kasryan committed Feb 4, 2023
1 parent 0e0e739 commit a5c8fb2
Show file tree
Hide file tree
Showing 3 changed files with 88 additions and 7 deletions.
79 changes: 76 additions & 3 deletions docs/08-flame-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,90 @@ make install
## Quickstart

### Configuring Brokers
As the flame system uses MQTT brokers to exchange messages during federated learning, to run the python library locally, you could either 1) install a local MQTT broker 2) use a public MQTT broker. Here we'll illustrate the second option.

Go to any examples that you wish to run locally in `examples` directory, change the `host` from `"flame-mosquitto"` to `broker.hivemq.com` in the `config.json` files of both the trainer and aggregator.
The following brokers are all for local testing.
If you wish to run federated learning accross multiple machines, please consider using the MQTT public broker.
This means setting `backend` and the `mqtt` broker in the config file as follows:

```json
"backend": "mqtt",
"brokers": [
{
"host": "broker.hivemq.com",
"sort": "mqtt"
}
]
```

However, this may lead to job ID collisions since it is a public broker.
Thus, for local testing, we recommend using either of the two options below.

#### Local MQTT Broker

Since the flame system uses MQTT brokers to exchange messages during federated learning, to run the python library locally, you may install a local MQTT broker as shown below.

```bash
sudo apt update
sudo apt install -y mosquitto
sudo systemctl status mosquitto
```

The last command should display something similar to this:

```bash
mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker
Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor pre>
Active: active (running) since Fri 2023-02-03 14:05:55 PST; 1h 20min ago
Docs: man:mosquitto.conf(5)
man:mosquitto(8)
Main PID: 75525 (mosquitto)
Tasks: 3 (limit: 9449)
Memory: 1.9M
CGroup: /system.slice/mosquitto.service
└─75525 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
```
That confirms that the mosquitto service is active.
From now on, you may use `sudo systemctl stop mosquitto` to stop the mosquitto service, `sudo systemctl start mosquitto` to start the service, and `sudo systemctl restart mosquitto` to restart the service.
Go ahead and change the two config files in `mnist/trainer` and `mnist/aggregator` to make sure `backend` is `mqtt`.
```json
"backend": "mqtt",
"brokers": [
{
"host": "localhost",
"sort": "mqtt"
},
{
"host": "localhost:10104",
"sort": "p2p"
}
]
```
Note that if you also want to use the local `mqtt` broker for other examples you should make sure that the `mqtt` broker has `host` set to `localhost`.
#### P2P
To start a `p2p` broker, go to the top `/flame` directory and run:
```bash
make install
cd ~
sudo ./.flame/bin/metaserver
```
After changing the two config files in `mnist/trainer` and `mnist/aggregator` so that `backend` is set to `p2p`, continue to the next section.
### Running an Example
In order to run this example, you will need to open two terminals.
In the first terminal, run the following command:
In the first terminal, run the following commands:
```bash
conda activate flame
cd examples/mnist/trainer
python keras/main.py config.json
Expand Down
8 changes: 6 additions & 2 deletions lib/python/flame/examples/mnist/aggregator/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,13 @@
"backend": "mqtt",
"brokers": [
{
"host": "broker.hivemq.com",
"host": "localhost",
"sort": "mqtt"
}
},
{
"host": "localhost:10104",
"sort": "p2p"
}
],
"channels": [
{
Expand Down
8 changes: 6 additions & 2 deletions lib/python/flame/examples/mnist/trainer/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,13 @@
"backend": "mqtt",
"brokers": [
{
"host": "broker.hivemq.com",
"host": "localhost",
"sort": "mqtt"
}
},
{
"host": "localhost:10104",
"sort": "p2p"
}
],
"channels": [
{
Expand Down

0 comments on commit a5c8fb2

Please sign in to comment.