Skip to content

Commit

Permalink
documenation for metaserver
Browse files Browse the repository at this point in the history
Documentation for using metaserver will allow users to run examples with a local broker.
This decreases the chances of any job ID collisions.

Modifications to the config.json for the mnist example were made in order to make it easier to switch to a local broker.
The readme does indicate how to do this for other examples now.
  • Loading branch information
kasryan committed Feb 3, 2023
1 parent ca690aa commit 7165cb5
Show file tree
Hide file tree
Showing 3 changed files with 50 additions and 6 deletions.
40 changes: 38 additions & 2 deletions docs/08-flame-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,44 @@ make install
## Quickstart

### Configuring Brokers
As the flame system uses MQTT brokers to exchange messages during federated learning, to run the python library locally, you could either 1) install a local MQTT broker 2) use a public MQTT broker. Here we'll illustrate the second option.
Since the flame system uses MQTT brokers to exchange messages during federated learning, to run the python library locally, you could either 1) install a local MQTT broker 2) use a public MQTT broker.

Go to any examples that you wish to run locally in `examples` directory, change the `host` from `"flame-mosquitto"` to `broker.hivemq.com` in the `config.json` files of both the trainer and aggregator.
Your choice should be specified in the section of the config file shown below.

```json
"backend": "p2p",
"brokers": [
{
"host": "broker.hivemq.com",
"sort": "mqtt"
},
{
"host": "localhost:10104",
"sort": "p2p"
}
]
```

If you wish to pursue the second option, you may go to any examples that you wish to run locally in `examples` directory and change the `host` from `"flame-mosquitto"` to `broker.hivemq.com` in the `config.json` files of both the trainer and aggregator.
Make sure that `backend` matches the `sort` of one of the brokers (for the mnist example, we already include `mqtt` and `p2p`).
Then, proceed to the next section to run the example.
However, this is not the best option since there may be job ID collisions with other users.
In order to avoid this, follow the first option's instructions below.

To start a local broker, go to the top `/flame` directory and run `make install`
After this, you can use `cd ~` to go to the HOME directory.
Now, to start metaserver, run `sudo ./.flame/bin/metaserver`.

After changing the two config files in `mnist/trainer` and `mnist/aggregator` so that `backend` is set to `p2p`, continue to the next section.

For future reference, you may have to add this to the list of brokers in the config file in order to use `metaserver`.

```json
{
"host": "localhost:10104",
"sort": "p2p"
}
```

### Running an Example

Expand All @@ -31,6 +66,7 @@ In order to run this example, you will need to open two terminals.
In the first terminal, run the following command:

```bash
conda activate flame
cd examples/mnist/trainer

python keras/main.py config.json
Expand Down
8 changes: 6 additions & 2 deletions lib/python/flame/examples/mnist/aggregator/config.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
{
"taskid": "49d06b7526964db86cf37c70e8e0cdb6bd7aa742",
"backend": "mqtt",
"backend": "p2p",
"brokers": [
{
"host": "broker.hivemq.com",
"sort": "mqtt"
}
},
{
"host": "localhost:10104",
"sort": "p2p"
}
],
"channels": [
{
Expand Down
8 changes: 6 additions & 2 deletions lib/python/flame/examples/mnist/trainer/config.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
{
"taskid": "505f9fc483cf4df68a2409257b5fad7d3c580370",
"backend": "mqtt",
"backend": "p2p",
"brokers": [
{
"host": "broker.hivemq.com",
"sort": "mqtt"
}
},
{
"host": "localhost:10104",
"sort": "p2p"
}
],
"channels": [
{
Expand Down

0 comments on commit 7165cb5

Please sign in to comment.