From 3417979d94c320d0ac471fada8aabf3b5d895339 Mon Sep 17 00:00:00 2001 From: mitchelbaker-cisa <149098823+mitchelbaker-cisa@users.noreply.github.com> Date: Mon, 12 Feb 2024 12:47:08 -0500 Subject: [PATCH] Fix deploy.sh data retention failure error (#190) * Fix deploysh data retention failure (#179) * Update deploy.sh * Update deploy.sh * Update deploy.sh * Update deploy.sh * Remove free (#188) * changed the word free to no-cost or no-cost to users * rephrased wording to 'which comes at no cost to users' --------- Co-authored-by: Linda Lovero-Waterhouse * Update upgrading.md with data retention failure resolution (#189) --------- Co-authored-by: Andrew Arz <149685528+aarz-snl@users.noreply.github.com> Co-authored-by: Linda Waterhouse <82845774+llwaterhouse@users.noreply.github.com> Co-authored-by: Linda Lovero-Waterhouse --- Chapter 3 Files/deploy.sh | 6 +++--- README.md | 2 +- docs/markdown/chapter3/chapter3.md | 2 +- docs/markdown/maintenance/upgrading.md | 19 ++++++++++++++++--- docs/markdown/prerequisites.md | 2 +- 5 files changed, 22 insertions(+), 9 deletions(-) diff --git a/Chapter 3 Files/deploy.sh b/Chapter 3 Files/deploy.sh index 0a25377e..a74b386d 100755 --- a/Chapter 3 Files/deploy.sh +++ b/Chapter 3 Files/deploy.sh @@ -535,13 +535,13 @@ function pipelineupdate() { function data_retention() { # Show ext4 disk - DF_OUTPUT="$(df -h -l -t ext4 --output=source,size /var/lib/docker)" + DF_OUTPUT="$(df -BG -l -t ext4 --output=source,size /var/lib/docker)" # Pull device name - DISK_DEV="$(echo "$DF_OUTPUT" | grep -Po '[0-9]+G')" + DISK_DEV="$(echo "$DF_OUTPUT" | awk 'NR==2 {print $1}')" # Pull device size - DISK_SIZE="${DISK_DEV/G/}" + DISK_SIZE="$(echo "$DF_OUTPUT" | awk 'NR==2 {print $2}' | sed 's/G//')" # Check if DISK_SIZE is empty or not a number if ! [[ "$DISK_SIZE" =~ ^[0-9]+$ ]]; then diff --git a/README.md b/README.md index 9075866b..cdcc4d95 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![Downloads](https://img.shields.io/github/downloads/cisagov/lme/total.svg)]() # Logging Made Easy -Initially created by NCSC and now maintained by CISA, Logging Made Easy is a self-install tutorial for small organizations to gain a basic level of centralized security logging for Windows clients and provide functionality to detect attacks. It's the coming together of multiple free and open software platforms, where LME helps the reader integrate them together to produce an end-to-end logging capability. We also provide some pre-made configuration files and scripts, although there is the option to do it on your own. +Initially created by NCSC and now maintained by CISA, Logging Made Easy is a self-install tutorial for small organizations to gain a basic level of centralized security logging for Windows clients and provide functionality to detect attacks. It's the coming together of multiple open software platforms which come at no cost to users, where LME helps the reader integrate them together to produce an end-to-end logging capability. We also provide some pre-made configuration files and scripts, although there is the option to do it on your own. Logging Made Easy can: - Show where administrative commands are being run on enrolled devices diff --git a/docs/markdown/chapter3/chapter3.md b/docs/markdown/chapter3/chapter3.md index a62ddcc0..c963ca22 100644 --- a/docs/markdown/chapter3/chapter3.md +++ b/docs/markdown/chapter3/chapter3.md @@ -15,7 +15,7 @@ In this chapter you will: This section covers the installation and configuration of the Database and search functionality on a Linux server. We will install the ‘ELK’ Stack from Elasticsearch for this portion. What is the ELK Stack? -"ELK" is the acronym for three free and open projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a "stash" like Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch. +"ELK" is the acronym for three open projects which come at no cost to users: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a "stash" like Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch. ![Elkstack components](/docs/imgs/elkstack.jpg)

diff --git a/docs/markdown/maintenance/upgrading.md b/docs/markdown/maintenance/upgrading.md index 12bae089..5f48ea70 100644 --- a/docs/markdown/maintenance/upgrading.md +++ b/docs/markdown/maintenance/upgrading.md @@ -119,15 +119,28 @@ This is a hotfix to address dashboards which failed to load on a fresh install o Please refer to the [Upgrading to latest version](/docs/markdown/maintenance/upgrading.md#upgrading-to-latest-version) to apply the hotfix. -## 6. Upgrade to latest version +## 6. v1.3.3 - Update on data retention failure during LME install + +This is a hotfix to address an error with data retention failure in the deploy.sh script during a fresh LME install. We recommend you upgrade to the latest version if you require disk sizes of 1TB or greater. + +If you've tried to install LME before, then run the following commands as root: +``` +git pull +git checkout main +cd /opt/lme/Chapter\ 3\ Files/ +sudo ./deploy.sh uninstall +sudo docker volume rm lme-esdata +sudo docker volume rm lme-logstashdata +sudo ./deploy.sh install +``` + +## 7. Upgrade to latest version To fetch the latest changes, on the Linux server, run the following commands as root: ``` git pull git checkout main cd /opt/lme/Chapter\ 3\ Files/ sudo ./deploy.sh uninstall -cd /opt/lme -cd Chapter\ 3\ Files/ sudo ./deploy.sh install ``` diff --git a/docs/markdown/prerequisites.md b/docs/markdown/prerequisites.md index fc54e515..f34e9ed0 100644 --- a/docs/markdown/prerequisites.md +++ b/docs/markdown/prerequisites.md @@ -29,7 +29,7 @@ Figure 1: High level overview, linking to documentation chapters The portions of this package developed by the United States government are distributed under the Creative Commons 0 ("CC0") license. Portions created by government contractors at the behest of CISA are provided with the explicit grant of right to use, modify, and redistribute the code subject to this statement and the existing license structure. All other portions, including new submissions from all others, are subject to the Apache License, Version 2.0. This project (scripts, documentation, and so on) is licensed under the [Apache License 2.0 and Creative Commons 0](../../LICENSE). -The design uses free and open software, we will maintain a pledge to ensure that no paid software licenses are needed above standard infrastructure costs (With the exception of Windows Operating system Licensing). +The design uses open software which comes at no cost to the user, we will maintain a pledge to ensure that no paid software licenses are needed above standard infrastructure costs (With the exception of Windows Operating system Licensing). You will need to pay for hosting, bandwidth and time; for an estimate of server specs that might be needed see this [blogpost from elasticsearch](https://www.elastic.co/blog/benchmarking-and-sizing-your-elasticsearch-cluster-for-logs-and-metrics). Then use your estimated server specs to determine a price for an on prem or cloud deployment.