Skip to content

Commit

Permalink
Documentation: added a doc for bbotl command-line.
Browse files Browse the repository at this point in the history
Signed-off-by: Ishan Tyagi <ishantyagi25@gmail.com>
  • Loading branch information
ishan16696 committed Jul 21, 2023
1 parent f697249 commit 4ef33dc
Showing 1 changed file with 317 additions and 0 deletions.
317 changes: 317 additions & 0 deletions cmd/bbolt/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,317 @@
# Introduction to bbolt command line

Bbolt is a tool for inspecting and examining bbolt databases by access the low level details of the bbolt database. `bbolt` is a command line client for Bbolt databases. <br>
To install bbolt command-line please refer [here](https://github.com/etcd-io/bbolt#installing).

**Note**: [Etcd](https://github.com/etcd-io/etcd) uses bbolt database as its backend storage engine. In this document, we will be
analysing etcd's bbolt database to inspect data in etcd cluster, if you don't want to use it then you can use [this](https://github.com/etcd-io/bbolt#opening-a-database) to create/open a bbolt database.

1. Start a single member etcd cluster with this command:
```
$etcd --data-dir default.etcd
```

It will create a `data-dir` with name `default.etcd` and directory structure will look like this:
```
~/tree default.etcd
default.etcd
└── member
├── snap
│   └── db --> this is bolt database file
└── wal
└── 0000000000000000-0000000000000000.wal
3 directories, 2 files
```

2. Put some dummy data using [etcdctl](https://github.com/etcd-io/etcd/tree/main/etcdctl).
3. Close the etcd to mimic as it got crashed and we will analyse the bbolt database using bbolt command line.

## Usage
- `bbolt command [arguments]`

### help
- help will print information about that command

```
~/bbolt help
The commands are:
bench run synthetic benchmark against bbolt
buckets print a list of buckets
check verifies integrity of bbolt database
compact copies a bbolt database, compacting it in the process
dump print a hexadecimal dump of a single page
get print the value of a key in a bucket
info print basic info
keys print a list of keys in a bucket
help print this screen
page print one or more pages in human readable format
pages print list of pages with their types
page-item print the key and value of a page item.
stats iterate over all pages and generate usage stats
surgery perform surgery on bbolt database
```

- you can use `help` with any command: `bbolt [command] -h` for more information about command.


## Analyse Bbolt database with bbolt command line

### info:
- print basic info about the given Bbolt database.
- usage:
`bbolt info [path to the bolt database]`

Example:
```
~/bbolt info ~/default.etcd/member/snap/db
Page Size: 4096
```

- **note**: page size is given in bytes
- Bbolt database is using page size of 4KB


### buckets:
- Buckets print a list of buckets of Bbolt database is currently having. Find more information on buckets: [here](https://github.com/etcd-io/bbolt#using-buckets)
- usage:
`bbolt buckets [path to the bolt database]`

Example:
```
bbolt buckets ~/default.etcd/member/snap/db
alarm
auth
authRoles
authUsers
cluster
key
lease
members
members_removed
meta
```

- It means when you start an etcd, it creates these `10` buckets using bbolt database.


### check:
- Check opens a database at given `[PATH]` and runs an exhaustive check to verify that all pages are accessible or are marked as freed. It also verifies that no pages are double referenced.
- usage:
`bbolt check [path to the bolt database]`

Example:
```
~/bbolt check ~/default.etcd/member/snap/db
ok
```

- It returns `ok` as our database file `db` is not corrupted.

### compact:
- Compact opens a database at given `[Source Path]` and walks it recursively, copying keys as they are found from all buckets, to a newly created database at `[Destination Path]`.
- usage:
`bbolt compact [options] -o [Destination Path] [Source Path]`

Example:
```
~/bbolt compact -o ~/db.compact ~/default.etcd/member/snap/db
16805888 -> 32768 bytes (gain=512.88x)
```
- It will create a compacted bbolt database file named `db.compact`.

### stats:
- To gather essential statistics about the bbolt database: `stats` performs an extensive search of the database to track every page reference. It starts at the current meta page and recursively iterates through every accessible bucket.
- usage:
`bbolt stats [path to the bolt database]`

Example:
```
~/bbolt stats ~/default.etcd/member/snap/db
Aggregate statistics for 10 buckets
Page count statistics
Number of logical branch pages: 0
Number of physical branch overflow pages: 0
Number of logical leaf pages: 0
Number of physical leaf overflow pages: 0
Tree statistics
Number of keys/value pairs: 11
Number of levels in B+tree: 1
Page size utilization
Bytes allocated for physical branch pages: 0
Bytes actually used for branch data: 0 (0%)
Bytes allocated for physical leaf pages: 0
Bytes actually used for leaf data: 0 (0%)
Bucket statistics
Total number of buckets: 10
Total number on inlined buckets: 10 (100%)
Bytes used for inlined buckets: 780 (0%)
```

### pages:
- Pages prints a table of pages with their type (meta, leaf, branch, freelist).
- The `meta` will store the metadata information of database.
- The `leaf` and `branch` pages will show a key count in the `items` column.
- The `freelist` will show the number of free pages, which are free for writing to again.
- The `overflow` column shows the number of blocks that the page spills over into.
- usage:
`bbolt pages [path to the bolt database]`

Example:
```
bbolt pages ~/default.etcd/member/snap/db
ID TYPE ITEMS OVRFLW
======== ========== ====== ======
0 meta 0
1 meta 0
2 free
3 leaf 10
4 freelist 2
5 free
```


### page:
- Page prints one or more pages in human readable format.
- usage:
```
bolt page [path to the bolt database] pageid [pageid...]
or: bolt page --all [path to the bolt database]
```

Example:
```
bbolt page ~/default.etcd/member/snap/db 3
Page ID: 3
Page Type: leaf
Total Size: 4096 bytes
Overflow pages: 0
Item Count: 10
"alarm": <pgid=0,seq=0>
"auth": <pgid=0,seq=0>
"authRoles": <pgid=0,seq=0>
"authUsers": <pgid=0,seq=0>
"cluster": <pgid=0,seq=0>
"key": <pgid=0,seq=0>
"lease": <pgid=0,seq=0>
"members": <pgid=0,seq=0>
"members_removed": <pgid=0,seq=0>
"meta": <pgid=0,seq=0>
```

- It prints information of pageID: `3`


### page-item:
- page-item prints a page item's key and value.
- usage:
```
bolt page-item [options] [path to the bolt database] <pageId> <itemId>
Additional options include:
--key-only
Print only the key
--value-only
Print only the value
--format
Output format. One of: auto|ascii-encoded|hex|bytes|redacted (default=ascii-encoded)
```

Example:
```
bbolt page-item --key-only ~/default.etcd/member/snap/db 3 7
"members"
```

- It returns the key as `--key-only` flag is passed of pageID: `3` and itemID: `7`

### dump:
- Dump prints a hexadecimal dump of one or more given pages.
- usage:
`bolt dump [path to the bolt database] [pageid...]`

### keys:
- Print a list of keys in the given bucket.
- usage:
`bolt keys [path to the bolt database] [BucketName]`

Example 1:
```
bbolt keys ~/default.etcd/member/snap/db meta
confState
consistent_index
term
```

- It list all the keys in `meta` bucket.

Example 2:
```
bbolt keys ~/Desktop/etcd_analysis/default.etcd/member/snap/db members
8e9e05c52164694d
```

- It list all the keys in `members` bucket which is a `memberId` of etcd cluster member.
- In this case we are running single member etcd cluster, hence only `1 memberId` is there. If we would have run a `3` member etcd cluster then it will return a `3 memberId` as `3 memberId` would have been present in `members` bucket.

### get:
- Print the value of the given key in the given bucket.
- usage:
`bolt get [path to the bolt database] [BucketName] key`


Example 1:
```
bbolt get --format=hex ~/Desktop/etcd_analysis/default.etcd/member/snap/db meta term
0000000000000004
```

- It returns the value of key: `term` in bucket `meta` in hexadecimal format.

Example 2:
```
bbolt get ~/Desktop/etcd_analysis/default.etcd/member/snap/db members 8e9e05c52164694d
{"id":10276657743932975437,"peerURLs":["http://localhost:2380"],"name":"default","clientURLs":["http://localhost:2379"]}
```

- It returns the value of key: `8e9e05c52164694d` in bucket `members`.

### bench:
- run synthetic benchmark against bbolt database.
- usage:
```
Usage:
-batch-size int
-blockprofile string
-count int
(default 1000)
-cpuprofile string
-fill-percent float
(default 0.5)
-key-size int
(default 8)
-memprofile string
-no-sync
-path string
-profile-mode string
(default "rw")
-read-mode string
(default "seq")
-value-size int
(default 32)
-work
-write-mode string
(default "seq")
```

0 comments on commit 4ef33dc

Please sign in to comment.