Skip to content

Commit

Permalink
Boltdb shipper doc fixes (#3265)
Browse files Browse the repository at this point in the history
* Remove incorrect value for boltdb_shipper.resync_interval in example doc which anyways has a default value of 5m

* Update incorrect default value for resync_interval mentioned in docs

* Add boltdb shipper configuration in config reference file
  • Loading branch information
sandeepsukhani authored Feb 26, 2021
1 parent ba25b0f commit faff8b6
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 2 deletions.
30 changes: 30 additions & 0 deletions docs/sources/configuration/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1353,6 +1353,36 @@ filesystem:
# CLI flag: -local.chunk-directory
directory: <string>
# Configures storing index in an Object Store(GCS/S3/Azure/Swift/Filesystem) in the form of boltdb files.
# Required fields only required when boltdb-shipper is defined in config.
boltdb_shipper:
# Directory where ingesters would write boltdb files which would then be
# uploaded by shipper to configured storage
# CLI flag: -boltdb.shipper.active-index-directory
[active_index_directory: <string> | default = ""]
# Shared store for keeping boltdb files. Supported types: gcs, s3, azure,
# filesystem
# CLI flag: -boltdb.shipper.shared-store
[shared_store: <string> | default = ""]
# Cache location for restoring boltDB files for queries
# CLI flag: -boltdb.shipper.cache-location
[cache_location: <string> | default = ""]
# TTL for boltDB files restored in cache for queries
# CLI flag: -boltdb.shipper.cache-ttl
[cache_ttl: <duration> | default = 24h]
# Resync downloaded files with the storage
# CLI flag: -boltdb.shipper.resync-interval
[resync_interval: <duration> | default = 5m]
# Number of days of index to be kept downloaded for queries. Works only with
# tables created with 24h period.
# CLI flag: -boltdb.shipper.query-ready-num-days
[query_ready_num_days: <int> | default = 0]
# Cache validity for active index entries. Should be no higher than
# the chunk_idle_period in the ingester settings.
# CLI flag: -store.index-cache-validity
Expand Down
1 change: 0 additions & 1 deletion docs/sources/configuration/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,6 @@ storage_config:
boltdb_shipper:
active_index_directory: /loki/index
cache_location: /loki/index_cache
resync_interval: 5s
shared_store: s3
aws:
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/operations/storage/boltdb-shipper.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ For all the queries which require chunks to be read from the store, Queriers als

Queriers lazily loads BoltDB files from shared object store to configured `cache_location`.
When a querier receives a read request, the query range from the request is resolved to period numbers and all the files for those period numbers are downloaded to `cache_location`, if not already.
Once we have downloaded files for a period we keep looking for updates in shared object store and download them every 15 Minutes by default.
Once we have downloaded files for a period we keep looking for updates in shared object store and download them every 5 Minutes by default.
Frequency for checking updates can be configured with `resync_interval` config.

To avoid keeping downloaded index files forever there is a ttl for them which defaults to 24 hours, which means if index files for a period are not used for 24 hours they would be removed from cache location.
Expand Down

0 comments on commit faff8b6

Please sign in to comment.