Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move and update docs #30

Merged
merged 1 commit into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,15 @@ We use [pre-commit](https://pre-commit.com/) as a tool to maintain a unified doc

Install

```sh
brew install pre-commit
```
```sh
brew install pre-commit
```

Run

```sh
pre-commit run -a
```
```sh
pre-commit run -a
```

## Deployment

Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ This tool ensures seamless access to ECR by handling credential rotation.
access:
secret:
name: ""
```
```

b) Download Helm-chart:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,76 +10,87 @@ We automated process of transferring data from the standalone version to the clu
Things that you have keep in mind before upgrading process:

- Migration time depends on resources(mostly on CPU and Disk IO) allocated for Clickhouse instances, during our tests we've mentioned average speed 50Gb per hour for instances with 4CPU/16Gb RAM

- It's necessary **not to set** flags as `--wait` and `--timeout` during upgrading process because of migration time

- Optional: We've developed a tool that checks consistency of data, you could use it before and after the upgrading process and compare results of those checks just to be sure that everything went as expected
Description of using `clickhouse-helper sanity`:
1) Gather required variables.

```shell
export ODM_URL=https://odm.localhost
export ODM_TOKEN=yourTokenFromRootUser
```
Description of using `clickhouse-helper sanity`:

1. Gather required variables.

```shell
export ODM_URL=https://odm.localhost
export ODM_TOKEN=yourTokenFromRootUser
```

2) Run check over ODM instance before the upgrade.
2. Run check over ODM instance before the upgrade.

```shell
docker run \
--env ODM_URL=${ODM_URL} \
--env ODM_TOKEN=${ODM_TOKEN} \
oci://091468197733.dkr.ecr.us-east-1.amazonaws.com/genestack/clickhouse-helper \
odm sanity > sanity-result-before.jsonl
```
```shell
docker run \
--env ODM_URL=${ODM_URL} \
--env ODM_TOKEN=${ODM_TOKEN} \
oci://091468197733.dkr.ecr.us-east-1.amazonaws.com/genestack/clickhouse-helper \
odm sanity > sanity-result-before.jsonl
```

3) Proceed with upgrading ODM instance
4) Run check over ODM instance after the upgrade
3. Proceed with upgrading ODM instance

```shell
docker run \
--env ODM_URL=${ODM_URL} \
--env ODM_TOKEN=${ODM_TOKEN} \
oci://091468197733.dkr.ecr.us-east-1.amazonaws.com/genestack/clickhouse-helper \
odm sanity > sanity-result-after.jsonl
```
4. Run check over ODM instance after the upgrade

5) Run a diff:
```shell
docker run \
--env ODM_URL=${ODM_URL} \
--env ODM_TOKEN=${ODM_TOKEN} \
oci://091468197733.dkr.ecr.us-east-1.amazonaws.com/genestack/clickhouse-helper \
odm sanity > sanity-result-after.jsonl
```

```shell
diff sanity-result-before.jsonl sanity-result-after.jsonl
```
5. Run a diff:

Basically there should be no changes, except studies with already uploaded VCF files, they should be started to reimport.
```shell
diff sanity-result-before.jsonl sanity-result-after.jsonl
```

Basically there should be no changes, except studies with already uploaded VCF files, they should be started to reimport.

## Version 1.57

!!! tip ""
Helm chart version 1.57.0

### Helm configuration changes

- Removed the link to the database for the service `func-file`. If you have it in your `values.yaml`, then you can safely remove the `spring` map completely.

```yaml
funcFile:
configurationFiles:
"application.yaml":
spring:
datasource:
# -- Mysql jdbc URL
url: "jdbc:mysql://..."
```
```yaml
funcFile:
configurationFiles:
"application.yaml":
spring:
datasource:
# -- Mysql jdbc URL
url: "jdbc:mysql://..."
```

- For the Clickhouse `busybox` image, the ability to set the repository and version has been added.

```yaml
clickhouse:
busyboxImage:
# -- Image registry
registry: docker.io
# -- Image repository
repository: busybox
# -- Image tag
tag: 1.36.1
```
```yaml
clickhouse:
busyboxImage:
# -- Image registry
registry: docker.io
# -- Image repository
repository: busybox
# -- Image tag
tag: 1.36.1
```

## Version 1.56

!!! tip ""
Helm chart version 1.56.1

### Export metrics to Genestack

Fluent-bit was introduced as an extra service tasked with collecting and dispatching metrics in Prometheus format to a Genestack.
Expand Down Expand Up @@ -120,6 +131,9 @@ global:

## Version 1.55

!!! tip ""
Helm chart version 1.55.4

### Configure ODM usage together with encrypted S3 bucket (SSE-KMS and SSE-S3 only)

#### Introduction
Expand Down
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Welcome to Open Data Manager Documentation
# Welcome to Open Data Manager Administrator Documentation

This documentation contains information how to use ODM system.
This documentation contains information how to deploy and support ODM system.
40 changes: 20 additions & 20 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,35 +1,35 @@
site_name: Open Data Manager Documentation
site_name: Open Data Manager Administrator Documentation
site_url: https://genestack.github.io/documentation

nav:
- Home: index.md
- Deployment:
- Home:
- index.md
- Requirements:
- Kubernetes: deployment/requirements/kubernetes.md
- Hardware: deployment/requirements/hardware.md
- External services: deployment/requirements/external-services.md
- Kubernetes: home/requirements/kubernetes.md
- Hardware: home/requirements/hardware.md
- External services: home/requirements/external-services.md
- Clouds:
- Amazon Web Services (AWS): deployment/clouds/aws.md
- Microsoft Azure: deployment/clouds/azure.md
- Amazon Web Services (AWS): home/clouds/aws.md
- Microsoft Azure: home/clouds/azure.md
- Single Sign-On:
- About SSO in ODM: deployment/single-sign-on/sso.md
- About SSO in ODM: home/single-sign-on/sso.md
- OpenID:
- Microsoft Azure: deployment/single-sign-on/openid/azure.md
- Google Workspace: deployment/single-sign-on/openid/google.md
- Okta: deployment/single-sign-on/openid/okta.md
- Microsoft Azure: home/single-sign-on/openid/azure.md
- Google Workspace: home/single-sign-on/openid/google.md
- Okta: home/single-sign-on/openid/okta.md
- SAML:
- Google Workspace: deployment/single-sign-on/saml/google.md
- Okta: deployment/single-sign-on/saml/okta.md
- Google Workspace: home/single-sign-on/saml/google.md
- Okta: home/single-sign-on/saml/okta.md
- SCIM:
- Microsoft Azure: deployment/single-sign-on/scim/azure.md
- Microsoft Azure: home/single-sign-on/scim/azure.md
- Helm:
- How to deploy: deployment/helm/how-to-deploy.md
- How to deploy: home/helm/how-to-deploy.md
- Troubleshooting:
- AWS S3: deployment/troubleshooting/aws-s3.md
- Azure SSO: deployment/troubleshooting/azure-sso.md
- Azure SCIM: deployment/troubleshooting/azure-scim.md
- AWS S3: home/troubleshooting/aws-s3.md
- Azure SSO: home/troubleshooting/azure-sso.md
- Azure SCIM: home/troubleshooting/azure-scim.md
- Release Notes:
- v1.50 - v1.59: deployment/release-notes/v1.50-v1.59.md
- v1.50 - v1.59: home/release-notes/v1.50-v1.59.md
theme:
name: material
palette:
Expand Down