Skip to content

Commit

Permalink
Merge pull request #457 from threefoldtech/development_update
Browse files Browse the repository at this point in the history
Updated Handle Bar with Dashboard
  • Loading branch information
scottyeager committed Mar 21, 2024
2 parents efa49ce + bf9c48f commit 74a6535
Show file tree
Hide file tree
Showing 22 changed files with 291 additions and 468 deletions.
6 changes: 2 additions & 4 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@
- [Grid Deployment](documentation/developers/grid_deployment/grid_deployment.md)
- [TFGrid Stacks](documentation/developers/grid_deployment/tfgrid_stacks.md)
- [Full VM Grid Deployment](documentation/developers/grid_deployment/grid_deployment_full_vm.md)
- [Guardians of the Grid](documentation/developers/grid_deployment/tfgrid_guardians.md)
- [Grid Snapshots](documentation/developers/grid_deployment/snapshots.md)
- [Farmers](documentation/farmers/farmers.md)
- [Build a 3Node](documentation/farmers/3node_building/3node_building.md)
- [1. Create a Farm](documentation/farmers/3node_building/1_create_farm.md)
Expand Down Expand Up @@ -341,11 +341,9 @@
- [3Bot Integration](knowledge_base/technology/smartcontract_it/smartcontract_3bot.md)
- [Farming](knowledge_base/farming/farming_toc.md)
- [Farming Rewards](knowledge_base/farming/farming_reward.md)
- [Proof-of-Utilization](knowledge_base/farming/proof_of_utilization.md)
- [Proof-of-Capacity](knowledge_base/farming/proof_of_capacity.md)
- [Proof-of-Utilization](knowledge_base/farming/proof_of_utilization.md)
- [PoC DAO Rules](knowledge_base/farming/poc_dao_rules.md)
- [Farming Reward v2](knowledge_base/farming/farming_reward2.md)
- [Farming Upgrade](knowledge_base/farming/farming_upgrade_2_3.md)
- [Cloud](knowledge_base/cloud/cloud_toc.md)
- [Cloud Units](knowledge_base/cloud/cloudunits.md)
- [Pricing](knowledge_base/cloud/pricing/pricing_toc.md)
Expand Down
2 changes: 1 addition & 1 deletion src/documentation/developers/developers.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,4 +87,4 @@ For complementary information on the technology developed by ThreeFold, refer to
- [Grid Deployment](./grid_deployment/grid_deployment.md)
- [TFGrid Stacks](./grid_deployment/tfgrid_stacks.md)
- [Full VM Grid Deployment](./grid_deployment/grid_deployment_full_vm.md)
- [Guardians of the Grid](./grid_deployment/tfgrid_guardians.md)
- [Grid Snapshots](./grid_deployment/snapshots.md)
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ The TFGrid whole source code is open-source and instances of the grid can be dep

- [TFGrid Stacks](./tfgrid_stacks.md)
- [Full VM Grid Deployment](./grid_deployment_full_vm.md)
- [Guardians of the Grid](./tfgrid_guardians.md)
- [Grid Snapshots](./snapshots.md)
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,4 @@ Go into the folder of the proper network, e.g. mainnet, and run the following co
```
git pull -r
docker compose --env-file .secrets.env --env-file .env up -d
```

If you're interested about hosting your own instance of the grid to strenghten the ThreeFold ecosystem, make sure to read the next section, [Guardians of the Grid](./tfgrid_guardians.md).
```
78 changes: 34 additions & 44 deletions src/documentation/developers/grid_deployment/snapshots.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,12 @@
<h2>Table of Contents</h2>

- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Services](#services)
- [ThreeFold Public Snapshots](#threefold-public-snapshots)
- [Deploy the Services with Scripts](#deploy-the-services-with-scripts)
- [Create the Snapshots](#create-the-snapshots)
- [Start All the Services](#start-all-the-services)
- [Stop All the Services](#stop-all-the-services)
- [Create the Snapshots](#create-the-snapshots)
- [Expose the Snapshots with Rsync](#expose-the-snapshots-with-rsync)
- [Create the Service Configuration File](#create-the-service-configuration-file)
- [Start the Service](#start-the-service)
Expand All @@ -19,12 +18,7 @@

To facilitate deploying grid backend services, we provide snapshots to significantly reduce sync time. This can be setup anywhere from scratch. Once all services are synced, one can use the scripts to create snapshots automatically.

## Prerequisites

There are a few prerequisites to properly run the ThreeFold services.

- [Docker engine](../computer_it_basics/docker_basics.md#install-docker-desktop-and-docker-engine)
- [Rsync](../computer_it_basics/file_transfer.md#rsync)
To learn how to deploy your own grid stack, read [this section](./grid_deployment_full_vm.md).

## Services

Expand Down Expand Up @@ -57,98 +51,96 @@ ThreeFold hosts all available snapshots at: [https://bknd.snapshot.grid.tf/](htt
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotsdev/processor-devnet-latest.tar.gz .
```

Let's now see how to use those snapshots to run the services via scripts.

## Deploy the Services with Scripts

You can deploy the 3 individual services using known methods such as [Docker](https://manual.grid.tf/computer_it_basics/docker_basics.html). To facilitate the process, scripts are provided that run the necessary docker commands.
You can deploy the 3 individual services using known methods such as [Docker](../../system_administrators/computer_it_basics/docker_basics.md). To facilitate the process, scripts are provided that run the necessary docker commands.

The first script creates the snapshots, while the second and third scripts serve to start and stop all services.

You can use the start script to start all services and then set a cron job to execute periodically the snapshot creation script. This will ensure that you always have the latest version available on your server.

### Start All the Services
### Create the Snapshots

You can start all services by running the provided scripts.
You can set a cron job to execute a script running rsync to create the snapshots and generate logs at a given interval.

- Download the script.
- First download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/startall.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/create_snapshot.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/startall.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/create_snapshot.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/startall.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/create_snapshot.sh
```
- Set the permissions of the script
```
chmod +x startall.sh
chmod +x create_snapshot.sh
```
- Run the script to start all services via docker engine.
- Make sure to a adjust the snapshot creation script for your specific deployment
- Set a cron job
```
./startall.sh
crontab -e
```
- Here is an example of a cron job where we execute the script every day at 1 AM and send the logs to `/var/log/snapshots/snapshots-cron.log`.
```sh
0 1 * * * sh /opt/snapshots/create-snapshot.sh > /var/log/snapshots/snapshots-cron.log 2>&1
```

### Stop All the Services
### Start All the Services

You can stop all services by running the provided scripts.
You can start all services by running the provided scripts.

- Download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/stopall.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/startall.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/stopall.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/startall.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/stopall.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/startall.sh
```
- Set the permissions of the script
```
chmod +x stopall.sh
chmod +x startall.sh
```
- Run the script to stop all services via docker engine.
- Run the script to start all services via docker engine.
```
./stopall.sh
./startall.sh
```

### Create the Snapshots
### Stop All the Services

You can set a cron job to execute a script running rsync to create the snapshots and generate logs at a given interval.
You can stop all services by running the provided scripts.

- First download the script.
- Download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/create_snapshot.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/stopall.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/create_snapshot.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/stopall.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/create_snapshot.sh
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/stopall.sh
```
- Set the permissions of the script
```
chmod +x create_snapshot.sh
chmod +x stopall.sh
```
- Make sure to a adjust the snapshot creation script for your specific deployment
- Set a cron job
- Run the script to stop all services via docker engine.
```
crontab -e
./stopall.sh
```
- Here is an example of a cron job where we execute the script every day at 1 AM and send the logs to `/var/log/snapshots/snapshots-cron.log`.
```sh
0 1 * * * sh /opt/snapshots/create-snapshot.sh > /var/log/snapshots/snapshots-cron.log 2>&1
```

## Expose the Snapshots with Rsync

Expand Down Expand Up @@ -202,5 +194,3 @@ systemctl start rsync
systemctl enable rsync
systemctl status rsync
```

If you're interested about hosting your own instance of the grid to strenghten the ThreeFold ecosystem, make sure to read the next section, [Guardians of the Grid](./tfgrid_guardians.md).
Loading

0 comments on commit 74a6535

Please sign in to comment.