Skip to content

Commit

Permalink
doc: adjust docker related desc in readme (#559)
Browse files Browse the repository at this point in the history 
* doc: add asf docker policy

* doc: adjust quick start order and others

* Update hugegraph-loader/README.md

* change desc

* change desc
  • Loading branch information
@aroundabout
aroundabout authored Dec 17, 2023
1 parent a63d182 commit 773469c
Show file tree
Hide file tree
Showing 3 changed files with 134 additions and 109 deletions.
4 changes: 2 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ it includes 5+ main modules.

## Usage

- [hugegraph-loader](./hugegraph-loader): We can use `docker run -itd --name loader hugegraph/loader` to quickly start [loader,](https://hub.docker.com/r/hugegraph/loader) or we can follow [this](./hugegraph-loader/README.md#212-docker-compose) to use docker-compose to start `loader` with `server`. And we can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/).
- [hugegraph-hubble](./hugegraph-hubble): We can use `docker run -itd --name=hubble -p 8088:8088 hugegraph/hubble` to quickly start [hubble,](https://hub.docker.com/r/hugegraph/hubble) or we can follow [this](hugegraph-hubble/README.md#quick-start) to use docker-compose to start `hubble` with `server`. And we can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/).
- [hugegraph-loader](./hugegraph-loader): We can follow the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/) to learn how to quickly start with `loader`.
- [hugegraph-hubble](./hugegraph-hubble): We can follow the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/) to learn how to quickly start with `hubble`.
- [hugegraph-client](./hugegraph-client): We can follow the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-client/) to learn how to quickly start with `client`.

## Maven Dependencies
Expand Down
106 changes: 82 additions & 24 deletions hugegraph-hubble/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,30 +16,88 @@ graph data load, schema management, graph relationship analysis, and graphical d

## Quick Start

We can quickly start `hubble` in two ways:

1. We can use `docker run -itd --name=hubble -p 8088:8088 hugegraph/hubble` to quickly start [hubble](https://hub.docker.com/r/hugegraph/hubble).
2. Or we can use the `docker-compose.yml` to start `hubble` with `hugegraph-server`. If we set `PRELOAD=true`, we can preload the example graph when starting `hugegraph-server`:

```yaml
version: '3'
services:
server:
image: hugegraph/hugegraph
container_name: graph
#environment:
# - PRELOAD=true
ports:
- 8080:8080

hubble:
image: hugegraph/hubble
container_name: hubble
ports:
- 8088:8088
```
Then we should follow the [hubble doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/#3platform-workflow) to create the graph.
There are three ways to get HugeGraph-Loader:

- Download the Toolchain binary package
- Source code compilation
- Use Docker image (Convenient for Test/Dev)

And you can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#2-get-hugegraph-loader)

### 1. Download the Toolchain binary package

`hubble` is in the `toolchain` project. First, download the binary tar tarball

```bash
wget https://downloads.apache.org/incubator/hugegraph/{version}/apache-hugegraph-toolchain-incubating-{version}.tar.gz
tar -xvf apache-hugegraph-toolchain-incubating-{version}.tar.gz
cd apache-hugegraph-toolchain-incubating-{version}.tar.gz/apache-hugegraph-hubble-incubating-{version}
```

Run `hubble`:

```
bin/start-hubble.sh
```

Then use a web browser to access `ip:8088` and you can see the `Hubble` page. You can stop the service using bin/stop-hubble.sh.

### 2. Clone source code then compile and install

> Note: Compiling Hubble requires the user’s local environment to have Node.js V16.x and yarn installed.
```bash
apt install curl build-essential
curl -o- https://github.com/raw/nvm-sh/nvm/v0.39.1/install.sh | bash
source ~/.bashrc
nvm install 16
```

Then, verify that the installed Node.js version is 16.x (please note that higher Node version may cause conflicts).

```bash
node -v
```

install `yarn` by the command below:

```bash
npm install -g yarn
```

Download the toolchain source code.

```bash
git clone https://github.com/apache/hugegraph-toolchain.git
```

Compile `hubble`. It depends on the loader and client, so you need to build these dependencies in advance during the compilation process (you can skip this step later).

```bash
cd incubator-hugegraph-toolchain
sudo pip install -r hugegraph-hubble/hubble-dist/assembly/travis/requirements.txt
mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp
cd hugegraph-hubble
mvn -e compile package -Dmaven.javadoc.skip=true -Dmaven.test.skip=true -ntp
cd apache-hugegraph-hubble-incubating*
```

Run `hubble`

```bash
bin/start-hubble.sh -d
```

### 3. User docker image (Convenient for Test/Dev)

We can use `docker run -itd --name=hubble -p 8088:8088 hugegraph/hubble` to quickly start [hubble](https://hub.docker.com/r/hugegraph/hubble). An you can visit [hubble deploy doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/#2-deploy) for more details.

Then we should follow the [hubble workflow doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-hubble/#3platform-workflow) to create the graph.

> Note:
> 1. The docker image of hugegraph-hubble is a convenience release, but not **official distribution** artifacts. You can find more details from [ASF Release Distribution Policy](https://infra.apache.org/release-distribution.html#dockerhub).
>
> 2. Recommand to use `release tag`(like `1.0.0`) for the stable version. Use `latest` tag to experience the newest functions in development.
## Doc

Expand Down
133 changes: 50 additions & 83 deletions hugegraph-loader/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,102 +15,77 @@ hugegraph-loader is a customizable command line utility for loading small to med
- Detecting schema from data automatically, reduce the complex work of schema management.
- Advanced customized operations with groovy script, users can configure how to construct vertices and edges by themselves.

## 2. Usage for Docker(Recommand)
## 2. Quick start

- Run `loader` with Docker
- Docker run
- Docker-compose
- Load data in docker container `loader`
There are three ways to get HugeGraph-Loader:

### 2.1 Start with Docker
- Download the compiled tarball
- Clone source code then compile and install
- Use docker image (Convenient for Test/Dev)

#### 2.1.1 Docker run
And you can find more details in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#2-get-hugegraph-loader)

Use the command `docker run -itd --name loader hugegraph/loader` to start loader.
### 2.1 Download the compiled tarball

If you want to load your data, you can mount the data folder like `-v /path/to/data/file:/loader/file`
Download the latest version of the HugeGraph-Toolchain release package:

``` bash
wget https://downloads.apache.org/incubator/hugegraph/{version}/apache-hugegraph-toolchain-incubating-{version}.tar.gz
tar zxf *hugegraph*.tar.gz
```

#### 2.1.2 Docker-compose
### 2.2 Clone source code then compile and install

The example `docker-compose.yml` is [here](./docker/example/docker-compose.yml)
Clone the latest version of HugeGraph-Loader source package:

If you want to load your data, you can mount the data folder like:
```yaml
volumes:
- /path/to/data/file:/loader/file
```bash
# 1. get from github
git clone https://github.com/apache/hugegraph-toolchain.git

# 2. get from direct (e.g. here is 1.0.0, please choose the latest version)
wget https://downloads.apache.org/incubator/hugegraph/{version}/apache-hugegraph-toolchain-incubating-{version}-src.tar.gz
```

Use the command `docker-compose up -d` to deploy `loader` with `server` and `hubble`.
Due to the license limitation of the `Oracle OJDBC`, you need to manually install ojdbc to the local maven repository. Visit the [Oracle jdbc downloads page](https://www.oracle.com/database/technologies/appdev/jdbc-drivers-archive.html). Select Oracle Database 12c Release 2 (12.2.0.1) drivers, as shown in the following figure.

### 2.2 Load data with docker container
After opening the link, select “ojdbc8.jar”.

#### 2.2.1 load data with docker

> If the `loader` and `server` is in the same docker network (for example, you deploy `loader` and `server` with `docker-compose`), we can set `-h {server_container_name}`. In our example, the container name of `server` is `graph`
>
> If `loader` is deployed alone, the `-h` should be set to the ip of the host of `server`. Other parameter description is [here](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#341-parameter-description)
Install ojdbc8 to the local maven repository, enter the directory where ojdbc8.jar is located, and execute the following command.

```bash
docker exec -it loader bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h graph -p 8080
mvn install:install-file -Dfile=./ojdbc8.jar -DgroupId=com.oracle -DartifactId=ojdbc8 -Dversion=12.2.0.1 -Dpackaging=jar
```

Then we can see the result.
Compile and generate tar package:

```bash
HugeGraphLoader worked in NORMAL MODE
vertices/edges loaded this time : 8/6
--------------------------------------------------
count metrics
input read success : 14
input read failure : 0
vertex parse success : 8
vertex parse failure : 0
vertex insert success : 8
vertex insert failure : 0
edge parse success : 6
edge parse failure : 0
edge insert success : 6
edge insert failure : 0
--------------------------------------------------
meter metrics
total time : 0.199s
read time : 0.046s
load time : 0.153s
vertex load time : 0.077s
vertex load rate(vertices/s) : 103
edge load time : 0.112s
edge load rate(edges/s) : 53
```
cd hugegraph-loader
mvn clean package -DskipTests
```

Then you can use `curl` or `hubble` to see the result.
### 2.3 Use docker image (Convenient for Test/Dev)

```bash
> curl "http://localhost:8080/graphs/hugegraph/graph/vertices" | gunzip
{"vertices":[{"id":1,"label":"software","type":"vertex","properties":{"name":"lop","lang":"java","price":328.0}},{"id":2,"label":"software","type":"vertex","properties":{"name":"ripple","lang":"java","price":199.0}},{"id":"1:tom","label":"person","type":"vertex","properties":{"name":"tom"}},{"id":"1:josh","label":"person","type":"vertex","properties":{"name":"josh","age":32,"city":"Beijing"}},{"id":"1:marko","label":"person","type":"vertex","properties":{"name":"marko","age":29,"city":"Beijing"}},{"id":"1:peter","label":"person","type":"vertex","properties":{"name":"peter","age":35,"city":"Shanghai"}},{"id":"1:vadas","label":"person","type":"vertex","properties":{"name":"vadas","age":27,"city":"Hongkong"}},{"id":"1:li,nary","label":"person","type":"vertex","properties":{"name":"li,nary","age":26,"city":"Wu,han"}}]}
```
#### 2.3.1 Docker run

If you want to check the edges, use `curl "http://localhost:8080/graphs/hugegraph/graph/edges" | gunzip`
Use the command `docker run -itd --name loader hugegraph/loader` to start loader.

#### 2.2.2 enter the docker container to load data
If you want to load your data, you can mount the data folder like `-v /path/to/data/file:/loader/file`

If you want to do some additional operation in the container, you can enter the container as follows:

```bash
docker exec -it loader bash
```
#### 2.3.2 Docker-compose

Then, you can load data as follows:
The example `docker-compose.yml` is [here](./docker/example/docker-compose.yml). Use the command `docker-compose up -d` to deploy `loader` with `server` and `hubble`.

```bash
sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h graph -p 8080
```
> Note:
> 1. The docker image of hugegraph-loader is a convenience release, not **official distribution** artifacts. You can find more details from [ASF Release Distribution Policy](https://infra.apache.org/release-distribution.html#dockerhub).
>
> 2. Recommand to use `release tag`(like `1.0.0`) for the stable version. Use `latest` tag to experience the newest functions in development.
The result is as same as above.
## 3 Load data

## 3. Use loader directly
### 3.1 Use loader directly

> notice: currently, version is `1.0.0`
> notice: currently, version is `1.2.0`
Download and unzip the compiled archive

Expand All @@ -129,31 +104,23 @@ sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/f

More details is in the [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/)

## 4. Building

You can also build the `loader` by yourself.

Required:
### 3.2 Load data with docker

- Java 8
- Maven 3.6+

To build without executing tests:
> If the `loader` and `server` is in the same docker network (for example, you deploy `loader` and `server` with `docker-compose`), we can set `-h {server_container_name}`. In our example, the container name of `server` is `graph`
>
> If `loader` is deployed alone, the `-h` should be set to the ip of the host of `server`. Other parameter description is [here](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#341-parameter-description)
Visit [doc](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/#45-use-docker-to-load-data) for more details.

```bash
mvn clean install -DskipTests=true
docker exec -it loader bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h graph -p 8080
```

To build with default tests:

```bash
mvn clean install
```

## 5. Doc
## 4. Doc

The [loader homepage](https://hugegraph.apache.org/docs/quickstart/hugegraph-loader/) contains more information about it.

## 6. License
## 5. License

hugegraph-loader is licensed under Apache 2.0 License.

0 comments on commit 773469c

Please sign in to comment.