Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elastisearch client in Relay server #31

Merged
merged 4 commits into from
Jul 10, 2023

Conversation

TheRealSibasishBehera
Copy link
Contributor

@TheRealSibasishBehera TheRealSibasishBehera commented May 15, 2023

Changes Made

Fixes for kubearmor/KubeArmor#836

  • Introduced Elasticsearch client for improved data indexing and search capabilities.
  • Updated manifests to enable the Elasticsearch service by default.
  • Added Kibana exportable dashboards for enhanced data visualization.
  • Added docs for using this feature

Signed-off-by: Sibasish Behera <fangedhamster3114@gmail.com>
Signed-off-by: Sibasish Behera <fangedhamster3114@gmail.com>
@TheRealSibasishBehera TheRealSibasishBehera marked this pull request as ready for review May 15, 2023 17:59
deployments/relay-deployment.yaml Outdated Show resolved Hide resolved
docs/ek-dashboards/ek-dashboards.md Outdated Show resolved Hide resolved
relay-server/elastisearch/adapter.go Outdated Show resolved Hide resolved
relay-server/main.go Outdated Show resolved Hide resolved
relay-server/elastisearch/adapter.go Outdated Show resolved Hide resolved
relay-server/elastisearch/adapter.go Outdated Show resolved Hide resolved
# storageClassName: ""
resources:
requests:
storage: 10Gi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need 10Gi? I am not sure what kind of memory do we require here.

Copy link
Contributor Author

@TheRealSibasishBehera TheRealSibasishBehera Jun 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

health status index     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana_1 UhiacNwdSyCdwrZ_WZOdRg   1   1         26            0     70.3kb         35.1kb
green  open   alert     nai7xLdcRaKe-xStBA44qQ   1   1          4            0     77.4kb         38.7kb

I have done rough calulations for 5Gi in each instance , with some real data here , taking a count of 1000 alerts per hour and assuming a storage capacity of total 12 GB(from 3 instances - 1Gb each for elasticsearch itself) , it can sustain approximately 27 days of storage , for above scenario.

Here each alert is taking ~20kb .

I am not sure how many alerts per hour is practical ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we expect the alerts to be less (otherwise something is really bad), I would say 1000 alerts/hr should be fine.
@nyrahul wdyt?

relay-server/elastisearch/adapter.go Outdated Show resolved Hide resolved
@Ankurk99
Copy link
Member

I couldn't see the resolved comments to be addressed, can you please take a look? Also, please resolve the merge conflicts.

@TheRealSibasishBehera
Copy link
Contributor Author

I couldn't see the resolved comments to be addressed, can you please take a look? Also, please resolve the merge conflicts.

i have resolved the suggestions comments and merge conflicts

Copy link
Member

@Ankurk99 Ankurk99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Signed-off-by: Sibasish Behera <fangedhamster3114@gmail.com>
Signed-off-by: Sibasish Behera <fangedhamster3114@gmail.com>
@daemon1024 daemon1024 merged commit 0ae1a8d into kubearmor:main Jul 10, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants