From 773469c069bc79e9e8159349a5b086f6da3f97eb Mon Sep 17 00:00:00 2001 From: Dandelion <49650772+aroundabout@users.noreply.github.com> Date: Sun, 17 Dec 2023 15:30:34 +0800 Subject: [PATCH] doc: adjust docker related desc in readme (#559) * doc: add asf docker policy * doc: adjust quick start order and others * Update hugegraph-loader/README.md * change desc * change desc --- README.md | 4 +- hugegraph-hubble/README.md | 106 ++++++++++++++++++++++------- hugegraph-loader/README.md | 133 ++++++++++++++----------------------- 3 files changed, 134 insertions(+), 109 deletions(-) diff --git a/README.md b/README.md index 8b04b7494..6c3053ff5 100644 --- a/README.md +++ b/README.md @@ -20,8 +20,8 @@ it includes 5+ main modules. ## Usage -- [hugegraph-loader](./hugegraph-loader): We can use `docker run -itd --name loader hugegraph/loader` to quickly start [loader,](https://hub.docker.com/r/hugegraph/loader) or we can follow [this](./hugegraph-loader/README.md#212-docker-compose) to use docker-compose to start `loader` with `server`. And we can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/). -- [hugegraph-hubble](./hugegraph-hubble): We can use `docker run -itd --name=hubble -p 8088:8088 hugegraph/hubble` to quickly start [hubble,](https://hub.docker.com/r/hugegraph/hubble) or we can follow [this](hugegraph-hubble/README.md#quick-start) to use docker-compose to start `hubble` with `server`. And we can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/). +- [hugegraph-loader](./hugegraph-loader): We can follow the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/) to learn how to quickly start with `loader`. +- [hugegraph-hubble](./hugegraph-hubble): We can follow the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/) to learn how to quickly start with `hubble`. - [hugegraph-client](./hugegraph-client): We can follow the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-client/) to learn how to quickly start with `client`. ## Maven Dependencies diff --git a/hugegraph-hubble/README.md b/hugegraph-hubble/README.md index 5faf67ed4..5b23c34ab 100644 --- a/hugegraph-hubble/README.md +++ b/hugegraph-hubble/README.md @@ -16,30 +16,88 @@ graph data load, schema management, graph relationship analysis, and graphical d ## Quick Start -We can quickly start `hubble` in two ways: - -1. We can use `docker run -itd --name=hubble -p 8088:8088 hugegraph/hubble` to quickly start [hubble](https://hub.docker.com/r/hugegraph/hubble). -2. Or we can use the `docker-compose.yml` to start `hubble` with `hugegraph-server`. If we set `PRELOAD=true`, we can preload the example graph when starting `hugegraph-server`: - - ```yaml - version: '3' - services: - server: - image: hugegraph/hugegraph - container_name: graph - #environment: - # - PRELOAD=true - ports: - - 8080:8080 - - hubble: - image: hugegraph/hubble - container_name: hubble - ports: - - 8088:8088 - ``` - -Then we should follow the [hubble doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/#3platform-workflow) to create the graph. +There are three ways to get HugeGraph-Loader: + +- Download the Toolchain binary package +- Source code compilation +- Use Docker image (Convenient for Test/Dev) + +And you can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#2-get-hugegraph-loader) + +### 1. Download the Toolchain binary package + +`hubble` is in the `toolchain` project. First, download the binary tar tarball + +```bash +wget https://downloads.apache.org/incubator/hugegraph/{version}/apache-hugegraph-toolchain-incubating-{version}.tar.gz +tar -xvf apache-hugegraph-toolchain-incubating-{version}.tar.gz +cd apache-hugegraph-toolchain-incubating-{version}.tar.gz/apache-hugegraph-hubble-incubating-{version} +``` + +Run `hubble`: + +``` +bin/start-hubble.sh +``` + +Then use a web browser to access `ip:8088` and you can see the `Hubble` page. You can stop the service using bin/stop-hubble.sh. + +### 2. Clone source code then compile and install + +> Note: Compiling Hubble requires the user’s local environment to have Node.js V16.x and yarn installed. + +```bash +apt install curl build-essential +curl -o- https://github.com/raw/nvm-sh/nvm/v0.39.1/install.sh | bash +source ~/.bashrc +nvm install 16 +``` + +Then, verify that the installed Node.js version is 16.x (please note that higher Node version may cause conflicts). + +```bash +node -v +``` + +install `yarn` by the command below: + +```bash +npm install -g yarn +``` + +Download the toolchain source code. + +```bash +git clone https://github.com/apache/hugegraph-toolchain.git +``` + +Compile `hubble`. It depends on the loader and client, so you need to build these dependencies in advance during the compilation process (you can skip this step later). + +```bash +cd incubator-hugegraph-toolchain +sudo pip install -r hugegraph-hubble/hubble-dist/assembly/travis/requirements.txt +mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp +cd hugegraph-hubble +mvn -e compile package -Dmaven.javadoc.skip=true -Dmaven.test.skip=true -ntp +cd apache-hugegraph-hubble-incubating* +``` + +Run `hubble` + +```bash +bin/start-hubble.sh -d +``` + +### 3. User docker image (Convenient for Test/Dev) + +We can use `docker run -itd --name=hubble -p 8088:8088 hugegraph/hubble` to quickly start [hubble](https://hub.docker.com/r/hugegraph/hubble). An you can visit [hubble deploy doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/#2-deploy) for more details. + +Then we should follow the [hubble workflow doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/#3platform-workflow) to create the graph. + +> Note: +> 1. The docker image of hugegraph-hubble is a convenience release, but not **official distribution** artifacts. You can find more details from [ASF Release Distribution Policy](https://infra.apache.org/release-distribution.html#dockerhub). +> +> 2. Recommand to use `release tag`(like `1.0.0`) for the stable version. Use `latest` tag to experience the newest functions in development. ## Doc diff --git a/hugegraph-loader/README.md b/hugegraph-loader/README.md index e81c2b2f1..6751da00a 100644 --- a/hugegraph-loader/README.md +++ b/hugegraph-loader/README.md @@ -15,102 +15,77 @@ hugegraph-loader is a customizable command line utility for loading small to med - Detecting schema from data automatically, reduce the complex work of schema management. - Advanced customized operations with groovy script, users can configure how to construct vertices and edges by themselves. -## 2. Usage for Docker(Recommand) +## 2. Quick start -- Run `loader` with Docker - - Docker run - - Docker-compose -- Load data in docker container `loader` +There are three ways to get HugeGraph-Loader: -### 2.1 Start with Docker +- Download the compiled tarball +- Clone source code then compile and install +- Use docker image (Convenient for Test/Dev) -#### 2.1.1 Docker run +And you can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#2-get-hugegraph-loader) -Use the command `docker run -itd --name loader hugegraph/loader` to start loader. +### 2.1 Download the compiled tarball -If you want to load your data, you can mount the data folder like `-v /path/to/data/file:/loader/file` +Download the latest version of the HugeGraph-Toolchain release package: +``` bash +wget https://downloads.apache.org/incubator/hugegraph/{version}/apache-hugegraph-toolchain-incubating-{version}.tar.gz +tar zxf *hugegraph*.tar.gz +``` -#### 2.1.2 Docker-compose +### 2.2 Clone source code then compile and install -The example `docker-compose.yml` is [here](./docker/example/docker-compose.yml) +Clone the latest version of HugeGraph-Loader source package: -If you want to load your data, you can mount the data folder like: -```yaml -volumes: - - /path/to/data/file:/loader/file +```bash +# 1. get from github +git clone https://github.com/apache/hugegraph-toolchain.git + +# 2. get from direct (e.g. here is 1.0.0, please choose the latest version) +wget https://downloads.apache.org/incubator/hugegraph/{version}/apache-hugegraph-toolchain-incubating-{version}-src.tar.gz ``` -Use the command `docker-compose up -d` to deploy `loader` with `server` and `hubble`. +Due to the license limitation of the `Oracle OJDBC`, you need to manually install ojdbc to the local maven repository. Visit the [Oracle jdbc downloads page](https://www.oracle.com/database/technologies/appdev/jdbc-drivers-archive.html). Select Oracle Database 12c Release 2 (12.2.0.1) drivers, as shown in the following figure. -### 2.2 Load data with docker container +After opening the link, select “ojdbc8.jar”. -#### 2.2.1 load data with docker - -> If the `loader` and `server` is in the same docker network (for example, you deploy `loader` and `server` with `docker-compose`), we can set `-h {server_container_name}`. In our example, the container name of `server` is `graph` -> -> If `loader` is deployed alone, the `-h` should be set to the ip of the host of `server`. Other parameter description is [here](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#341-parameter-description) +Install ojdbc8 to the local maven repository, enter the directory where ojdbc8.jar is located, and execute the following command. ```bash -docker exec -it loader bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h graph -p 8080 +mvn install:install-file -Dfile=./ojdbc8.jar -DgroupId=com.oracle -DartifactId=ojdbc8 -Dversion=12.2.0.1 -Dpackaging=jar ``` -Then we can see the result. +Compile and generate tar package: -```bash -HugeGraphLoader worked in NORMAL MODE -vertices/edges loaded this time : 8/6 --------------------------------------------------- -count metrics - input read success : 14 - input read failure : 0 - vertex parse success : 8 - vertex parse failure : 0 - vertex insert success : 8 - vertex insert failure : 0 - edge parse success : 6 - edge parse failure : 0 - edge insert success : 6 - edge insert failure : 0 --------------------------------------------------- -meter metrics - total time : 0.199s - read time : 0.046s - load time : 0.153s - vertex load time : 0.077s - vertex load rate(vertices/s) : 103 - edge load time : 0.112s - edge load rate(edges/s) : 53 +``` +cd hugegraph-loader +mvn clean package -DskipTests ``` -Then you can use `curl` or `hubble` to see the result. +### 2.3 Use docker image (Convenient for Test/Dev) -```bash -> curl "http://localhost:8080/graphs/hugegraph/graph/vertices" | gunzip -{"vertices":[{"id":1,"label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328.0}},{"id":2,"label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199.0}},{"id":"1:tom","label":"person","type":"vertex","properties":{"name":"tom"}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"1:li,nary","label":"person","type":"vertex","properties":{"name":"li,nary","age":26,"city":"Wu,han"}}]} -``` +#### 2.3.1 Docker run -If you want to check the edges, use `curl "http://localhost:8080/graphs/hugegraph/graph/edges" | gunzip` +Use the command `docker run -itd --name loader hugegraph/loader` to start loader. -#### 2.2.2 enter the docker container to load data +If you want to load your data, you can mount the data folder like `-v /path/to/data/file:/loader/file` -If you want to do some additional operation in the container, you can enter the container as follows: -```bash -docker exec -it loader bash -``` +#### 2.3.2 Docker-compose -Then, you can load data as follows: +The example `docker-compose.yml` is [here](./docker/example/docker-compose.yml). Use the command `docker-compose up -d` to deploy `loader` with `server` and `hubble`. -```bash -sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h graph -p 8080 -``` +> Note: +> 1. The docker image of hugegraph-loader is a convenience release, not **official distribution** artifacts. You can find more details from [ASF Release Distribution Policy](https://infra.apache.org/release-distribution.html#dockerhub). +> +> 2. Recommand to use `release tag`(like `1.0.0`) for the stable version. Use `latest` tag to experience the newest functions in development. -The result is as same as above. +## 3 Load data -## 3. Use loader directly +### 3.1 Use loader directly -> notice: currently, version is `1.0.0` +> notice: currently, version is `1.2.0` Download and unzip the compiled archive @@ -129,31 +104,23 @@ sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/f More details is in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/) -## 4. Building - -You can also build the `loader` by yourself. - -Required: +### 3.2 Load data with docker -- Java 8 -- Maven 3.6+ - -To build without executing tests: +> If the `loader` and `server` is in the same docker network (for example, you deploy `loader` and `server` with `docker-compose`), we can set `-h {server_container_name}`. In our example, the container name of `server` is `graph` +> +> If `loader` is deployed alone, the `-h` should be set to the ip of the host of `server`. Other parameter description is [here](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#341-parameter-description) +Visit [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#45-use-docker-to-load-data) for more details. + ```bash -mvn clean install -DskipTests=true +docker exec -it loader bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h graph -p 8080 ``` -To build with default tests: - -```bash -mvn clean install -``` -## 5. Doc +## 4. Doc The [loader homepage](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/) contains more information about it. -## 6. License +## 5. License hugegraph-loader is licensed under Apache 2.0 License.