From 4ef33dc9257a40672529598d56caf653156eeda8 Mon Sep 17 00:00:00 2001 From: Ishan Tyagi Date: Fri, 21 Jul 2023 16:53:15 +0530 Subject: [PATCH] Documentation: added a doc for bbotl command-line. Signed-off-by: Ishan Tyagi --- cmd/bbolt/README.md | 317 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 317 insertions(+) create mode 100644 cmd/bbolt/README.md diff --git a/cmd/bbolt/README.md b/cmd/bbolt/README.md new file mode 100644 index 000000000..75be11743 --- /dev/null +++ b/cmd/bbolt/README.md @@ -0,0 +1,317 @@ +# Introduction to bbolt command line + +Bbolt is a tool for inspecting and examining bbolt databases by access the low level details of the bbolt database. `bbolt` is a command line client for Bbolt databases.
+To install bbolt command-line please refer [here](https://github.com/etcd-io/bbolt#installing). + +**Note**: [Etcd](https://github.com/etcd-io/etcd) uses bbolt database as its backend storage engine. In this document, we will be +analysing etcd's bbolt database to inspect data in etcd cluster, if you don't want to use it then you can use [this](https://github.com/etcd-io/bbolt#opening-a-database) to create/open a bbolt database. + +1. Start a single member etcd cluster with this command: + ``` + $etcd --data-dir default.etcd + ``` + + It will create a `data-dir` with name `default.etcd` and directory structure will look like this: + ``` + ~/tree default.etcd + default.etcd + └── member + ├── snap + │   └── db --> this is bolt database file + └── wal + └── 0000000000000000-0000000000000000.wal + + 3 directories, 2 files + ``` + +2. Put some dummy data using [etcdctl](https://github.com/etcd-io/etcd/tree/main/etcdctl). +3. Close the etcd to mimic as it got crashed and we will analyse the bbolt database using bbolt command line. + +## Usage + - `bbolt command [arguments]` + +### help + - help will print information about that command + + ``` + ~/bbolt help + + The commands are: + + bench run synthetic benchmark against bbolt + buckets print a list of buckets + check verifies integrity of bbolt database + compact copies a bbolt database, compacting it in the process + dump print a hexadecimal dump of a single page + get print the value of a key in a bucket + info print basic info + keys print a list of keys in a bucket + help print this screen + page print one or more pages in human readable format + pages print list of pages with their types + page-item print the key and value of a page item. + stats iterate over all pages and generate usage stats + surgery perform surgery on bbolt database + ``` + +- you can use `help` with any command: `bbolt [command] -h` for more information about command. + + +## Analyse Bbolt database with bbolt command line + +### info: + - print basic info about the given Bbolt database. + - usage: + `bbolt info [path to the bolt database]` + + Example: + ``` + ~/bbolt info ~/default.etcd/member/snap/db + Page Size: 4096 + ``` + + - **note**: page size is given in bytes + - Bbolt database is using page size of 4KB + + +### buckets: + - Buckets print a list of buckets of Bbolt database is currently having. Find more information on buckets: [here](https://github.com/etcd-io/bbolt#using-buckets) + - usage: + `bbolt buckets [path to the bolt database]` + + Example: + ``` + bbolt buckets ~/default.etcd/member/snap/db + alarm + auth + authRoles + authUsers + cluster + key + lease + members + members_removed + meta + ``` + + - It means when you start an etcd, it creates these `10` buckets using bbolt database. + + +### check: + - Check opens a database at given `[PATH]` and runs an exhaustive check to verify that all pages are accessible or are marked as freed. It also verifies that no pages are double referenced. + - usage: + `bbolt check [path to the bolt database]` + + Example: + ``` + ~/bbolt check ~/default.etcd/member/snap/db + ok + ``` + + - It returns `ok` as our database file `db` is not corrupted. + +### compact: + - Compact opens a database at given `[Source Path]` and walks it recursively, copying keys as they are found from all buckets, to a newly created database at `[Destination Path]`. + - usage: + `bbolt compact [options] -o [Destination Path] [Source Path]` + + Example: + ``` + ~/bbolt compact -o ~/db.compact ~/default.etcd/member/snap/db + 16805888 -> 32768 bytes (gain=512.88x) + ``` + - It will create a compacted bbolt database file named `db.compact`. + +### stats: + - To gather essential statistics about the bbolt database: `stats` performs an extensive search of the database to track every page reference. It starts at the current meta page and recursively iterates through every accessible bucket. + - usage: + `bbolt stats [path to the bolt database]` + + Example: + ``` + ~/bbolt stats ~/default.etcd/member/snap/db + Aggregate statistics for 10 buckets + + Page count statistics + Number of logical branch pages: 0 + Number of physical branch overflow pages: 0 + Number of logical leaf pages: 0 + Number of physical leaf overflow pages: 0 + Tree statistics + Number of keys/value pairs: 11 + Number of levels in B+tree: 1 + Page size utilization + Bytes allocated for physical branch pages: 0 + Bytes actually used for branch data: 0 (0%) + Bytes allocated for physical leaf pages: 0 + Bytes actually used for leaf data: 0 (0%) + Bucket statistics + Total number of buckets: 10 + Total number on inlined buckets: 10 (100%) + Bytes used for inlined buckets: 780 (0%) + ``` + +### pages: + - Pages prints a table of pages with their type (meta, leaf, branch, freelist). + - The `meta` will store the metadata information of database. + - The `leaf` and `branch` pages will show a key count in the `items` column. + - The `freelist` will show the number of free pages, which are free for writing to again. + - The `overflow` column shows the number of blocks that the page spills over into. + - usage: + `bbolt pages [path to the bolt database]` + + Example: + ``` + bbolt pages ~/default.etcd/member/snap/db + ID TYPE ITEMS OVRFLW + ======== ========== ====== ====== + 0 meta 0 + 1 meta 0 + 2 free + 3 leaf 10 + 4 freelist 2 + 5 free + ``` + + +### page: + - Page prints one or more pages in human readable format. + - usage: + ``` + bolt page [path to the bolt database] pageid [pageid...] + or: bolt page --all [path to the bolt database] + ``` + + Example: + ``` + bbolt page ~/default.etcd/member/snap/db 3 + Page ID: 3 + Page Type: leaf + Total Size: 4096 bytes + Overflow pages: 0 + Item Count: 10 + + "alarm": + "auth": + "authRoles": + "authUsers": + "cluster": + "key": + "lease": + "members": + "members_removed": + "meta": + ``` + + - It prints information of pageID: `3` + + +### page-item: + - page-item prints a page item's key and value. + - usage: + ``` + bolt page-item [options] [path to the bolt database] + Additional options include: + + --key-only + Print only the key + --value-only + Print only the value + --format + Output format. One of: auto|ascii-encoded|hex|bytes|redacted (default=ascii-encoded) + ``` + + Example: + ``` + bbolt page-item --key-only ~/default.etcd/member/snap/db 3 7 + "members" + ``` + + - It returns the key as `--key-only` flag is passed of pageID: `3` and itemID: `7` + +### dump: + - Dump prints a hexadecimal dump of one or more given pages. + - usage: + `bolt dump [path to the bolt database] [pageid...]` + +### keys: + - Print a list of keys in the given bucket. + - usage: + `bolt keys [path to the bolt database] [BucketName]` + + Example 1: + ``` + bbolt keys ~/default.etcd/member/snap/db meta + confState + consistent_index + term + ``` + + - It list all the keys in `meta` bucket. + + Example 2: + ``` + bbolt keys ~/Desktop/etcd_analysis/default.etcd/member/snap/db members + 8e9e05c52164694d + ``` + + - It list all the keys in `members` bucket which is a `memberId` of etcd cluster member. + - In this case we are running single member etcd cluster, hence only `1 memberId` is there. If we would have run a `3` member etcd cluster then it will return a `3 memberId` as `3 memberId` would have been present in `members` bucket. + +### get: + - Print the value of the given key in the given bucket. + - usage: + `bolt get [path to the bolt database] [BucketName] key` + + + Example 1: + ``` + bbolt get --format=hex ~/Desktop/etcd_analysis/default.etcd/member/snap/db meta term + 0000000000000004 + ``` + + - It returns the value of key: `term` in bucket `meta` in hexadecimal format. + + Example 2: + ``` + bbolt get ~/Desktop/etcd_analysis/default.etcd/member/snap/db members 8e9e05c52164694d + {"id":10276657743932975437,"peerURLs":["http://localhost:2380"],"name":"default","clientURLs":["http://localhost:2379"]} + ``` + + - It returns the value of key: `8e9e05c52164694d` in bucket `members`. + +### bench: + - run synthetic benchmark against bbolt database. + - usage: + ``` + Usage: + -batch-size int + + -blockprofile string + + -count int + (default 1000) + -cpuprofile string + + -fill-percent float + (default 0.5) + -key-size int + (default 8) + -memprofile string + + -no-sync + + -path string + + -profile-mode string + (default "rw") + -read-mode string + (default "seq") + -value-size int + (default 32) + -work + + -write-mode string + (default "seq") + ``` +