Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update common version and remove jersey code #538

Merged
merged 60 commits into from
Dec 5, 2023
Merged
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
884166c
Update client-ci.yml
zhenyuT Aug 13, 2023
cf729d5
replace jersey with okhttp
Aug 14, 2023
74f92dc
Merge remote-tracking branch 'origin/master'
Aug 14, 2023
3c10111
fix test error
Aug 14, 2023
1eabd9a
fix the problem that mockito can't mock final class
Aug 14, 2023
ab2f917
fix error
Aug 14, 2023
3fe0244
Update BaseClientTest.java
zhenyuT Aug 15, 2023
cb79e56
https support && add licence header
Aug 16, 2023
552a108
fix licence check error
Aug 16, 2023
9d211f8
fix error
Aug 16, 2023
a9c6c88
fix licence check error
Aug 16, 2023
bd5e5e7
fix error
Aug 17, 2023
277fe04
rename rest class && remove jersey dependency
Aug 17, 2023
1975bfd
add workflow_dispatch
Aug 17, 2023
ae9de53
add workflow_dispatch
Aug 17, 2023
41be461
Merge branch 'master' into release-removejersey
Aug 17, 2023
ecc5580
hubble error fix
Aug 17, 2023
5740701
update okhttp version of hubble
Aug 17, 2023
ef7846f
refactor class name
Aug 17, 2023
aff4455
add workflow_dispatch
Aug 17, 2023
d3d3daf
fix licence dependency check error
Aug 17, 2023
57a2144
Merge remote-tracking branch 'origin/master'
zhenyuT Nov 22, 2023
389431a
feat: merge request
zhenyuT Nov 22, 2023
93f83ab
chore(ci): add stage profile settings
zhenyuT Nov 22, 2023
a314397
Merge branch 'visit-stage'
zhenyuT Nov 22, 2023
9c52320
update common version and fix code
zhenyuT Nov 22, 2023
6bc34d6
fix
zhenyuT Nov 22, 2023
b855f62
Merge branch 'visit-stage'
zhenyuT Nov 22, 2023
c637fcd
fix
zhenyuT Nov 22, 2023
f45f176
update version
zhenyuT Nov 22, 2023
150bb0e
fix spark-connector-ci
zhenyuT Nov 22, 2023
75ff6d1
fix spark-connector-ci
zhenyuT Nov 22, 2023
57e4d76
fix spark-connector-ci
zhenyuT Nov 22, 2023
fcd0735
update client version
zhenyuT Nov 22, 2023
82b401a
update client version
zhenyuT Nov 22, 2023
164a50a
Merge remote-tracking branch 'upstream/master'
zhenyuT Nov 23, 2023
ea06f2c
fix license-checker
zhenyuT Nov 23, 2023
49a2343
code optimize
zhenyuT Nov 23, 2023
f0db02a
remove useless dependency
zhenyuT Nov 23, 2023
e593008
remove useless code
zhenyuT Nov 23, 2023
3eb6c42
fix licence check
zhenyuT Nov 23, 2023
cda3e28
fix import & update api version
imbajin Nov 24, 2023
e03074c
remove redundant common import in hubble
imbajin Nov 24, 2023
76a85ee
remove redundant lombok version
zhenyuT Nov 24, 2023
cdd1012
remove useless code
zhenyuT Nov 24, 2023
30396fb
add comment
zhenyuT Nov 24, 2023
d9603c0
rename id
zhenyuT Nov 25, 2023
0a09aac
add encode test case
zhenyuT Nov 26, 2023
3f8b42d
add encode test case and fix code issue
zhenyuT Nov 27, 2023
7e2f5ae
fix test error
zhenyuT Nov 28, 2023
bba55b8
fix test error
zhenyuT Nov 28, 2023
acc65e0
fix code issue
zhenyuT Nov 29, 2023
bb79524
fix code issue
zhenyuT Nov 29, 2023
4441ac8
fix code issue
zhenyuT Nov 29, 2023
0d215a4
update hg server commitId
zhenyuT Nov 29, 2023
e65644b
debug ci error
zhenyuT Nov 30, 2023
142192a
revert commit ID & let it pass 100
imbajin Dec 4, 2023
a18802b
Update client-ci.yml
imbajin Dec 4, 2023
4f1622c
Merge branch 'master' into master
imbajin Dec 5, 2023
a3b4423
tiny fix
imbajin Dec 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/client-ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: "hugegraph-client-ci"

on:
workflow_dispatch:
push:
branches:
- master
Expand All @@ -23,11 +24,11 @@ jobs:
env:
USE_STAGE: 'true' # Whether to include the stage repository.
TRAVIS_DIR: hugegraph-client/assembly/travis
COMMIT_ID: be6ee386b9939dc6bd6fcbdf2274b8acc3a0a314
COMMIT_ID: 69e6b461add1051831dd6cc0c36a5e249a3b3176
strategy:
fail-fast: false
matrix:
JAVA_VERSION: ['8']
JAVA_VERSION: [ '8' ]
steps:
- name: Install JDK 8
uses: actions/setup-java@v3
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
name: "CodeQL"

on:
workflow_dispatch:
push:
branches: [ master, release-* ]
pull_request:
# The branches below must be a subset of the branches above
# branches: [ master ] # enable in all PR
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/hubble-ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: "hugegraph-hubble-ci"

on:
workflow_dispatch:
push:
branches:
- master
Expand All @@ -23,7 +24,7 @@ on:
env:
TRAVIS_DIR: hugegraph-hubble/hubble-dist/assembly/travis
# TODO: need update it later (eed6103359fe40d2f1476fb8c56d9388c3111a99)
COMMIT_ID: be6ee386b9939dc6bd6fcbdf2274b8acc3a0a314
COMMIT_ID: 69e6b461add1051831dd6cc0c36a5e249a3b3176
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imbajin can we share the same COMMIT_ID with client-ci.yml (just use an global env var)

Copy link
Member

@imbajin imbajin Dec 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imbajin can we share the same COMMIT_ID with client-ci.yml (just use an global env var)

I do want it too but not found the way to achieve it, however I get a better way to avoid hard code commit ID in ci

we could use "latestCommitID - X" (X >= 15 maybe) to get the stable code in server & no need update the ID by ourselves if not necessary (will submit a PR after this adaptor) #542

Copy link
Contributor

@javeme javeme Dec 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a method to use latestCommitID - X, but we can't control it freely. If we want to adapt to the latest code, we still need to change the X.
we may try reusable-workflow: https://docs.github.com/en/actions/using-workflows/reusing-workflows#using-inputs-and-secrets-in-a-reusable-workflow

Copy link
Member

@imbajin imbajin Dec 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a method to use latestCommitID - X, but we can't control it freely. If we want to adapt to the latest code, we still need to change the X. we may try reusable-workflow: docs.github.com/en/actions/using-workflows/reusing-workflows#using-inputs-and-secrets-in-a-reusable-workflow

Get it, but we can't set secrets/variables in settings, we lack the permission to set it 😢 (This is also the reason why in the early code, we could only write fixed commitID in multiple CI files)

BTW, we could create/submit a common action and publish it to github action repo & let other actions to extend it if github supports


jobs:
hubble-ci:
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/loader-ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: "hugegraph-loader-ci"

on:
workflow_dispatch:
push:
branches:
- master
Expand All @@ -25,7 +26,7 @@ jobs:
USE_STAGE: 'true' # Whether to include the stage repository.
TRAVIS_DIR: hugegraph-loader/assembly/travis
STATIC_DIR: hugegraph-loader/assembly/static
COMMIT_ID: be6ee386b9939dc6bd6fcbdf2274b8acc3a0a314
COMMIT_ID: 69e6b461add1051831dd6cc0c36a5e249a3b3176
DB_USER: root
DB_PASS: root
DB_DATABASE: load_test
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/tools-ci.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
name: "hugegraph-tools-ci"
on:
workflow_dispatch:
push:
branches:
- master
Expand All @@ -24,7 +25,7 @@ jobs:
USE_STAGE: 'true' # Whether to include the stage repository.
TRAVIS_DIR: hugegraph-tools/assembly/travis
# TODO: could we use one param to unify it? or use a action template (could use one ci file)
COMMIT_ID: be6ee386b9939dc6bd6fcbdf2274b8acc3a0a314
COMMIT_ID: 69e6b461add1051831dd6cc0c36a5e249a3b3176
steps:
- name: Install JDK 11
uses: actions/setup-java@v3
Expand Down
1 change: 1 addition & 0 deletions .licenserc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ header: # `header` section is configurations for source codes license header.
- 'assembly/**'
- '.github/**/*'
- '**/target/*'
- 'hugegraph-client/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker'
# - 'hugegraph-hubble/hubble-fe/**'
comment: on-failure # on what condition license-eye will comment on the pull request, `on-failure`, `always`, `never`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,5 +60,5 @@ echo "gremlinserver.url=http://127.0.0.1:8282" >> ${REST_SERVER_CONFIG}

# start HugeGraphServer with https protocol
bin/init-store.sh
bin/start-hugegraph.sh
bin/start-hugegraph.sh || (cat logs/hugegraph-server.log && exit 1)
cd ../
19 changes: 9 additions & 10 deletions hugegraph-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
License for the specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
Expand All @@ -30,15 +30,12 @@
<packaging>jar</packaging>

<name>${project.artifactId}</name>
<url>https://github.com/apache/incubator-hugegraph-toolchain/tree/master/hugegraph-client</url>
<url>https://github.com/apache/hugegraph-toolchain/tree/master/hugegraph-client</url>
<description>
hugegraph-client is a Java-written client of HugeGraph, providing
operations of graph, schema, gremlin, variables and traversals etc.
</description>

<properties>
</properties>

<dependencies>
<dependency>
<groupId>org.apache.hugegraph</groupId>
Expand All @@ -48,15 +45,17 @@
<groupId>org.lz4</groupId>
<artifactId>lz4-java</artifactId>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet</artifactId>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
<optional>true</optional>
</dependency>
zhenyuT marked this conversation as resolved.
Show resolved Hide resolved
</dependencies>

<build>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ public User get(Object id) {
}

public UserRole getUserRole(Object id) {
String idEncoded = RestClient.encode(formatEntityId(id));
String path = String.join("/", this.path(), idEncoded, "role");
String formattedId = formatEntityId(id);
String path = String.join("/", this.path(), formattedId, "role");
RestResult result = this.client.get(path);
return result.readObject(UserRole.class);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@

import org.apache.hugegraph.client.RestClient;
import org.apache.hugegraph.exception.NotAllCreatedException;
import org.apache.hugegraph.rest.RestHeaders;
import org.apache.hugegraph.rest.RestResult;
import org.apache.hugegraph.structure.constant.Direction;
import org.apache.hugegraph.structure.constant.HugeType;
Expand All @@ -32,8 +33,6 @@

import com.google.common.collect.ImmutableMap;

import jakarta.ws.rs.core.MultivaluedHashMap;

public class EdgeAPI extends GraphAPI {

public EdgeAPI(RestClient client, String graph) {
Expand All @@ -51,12 +50,9 @@ public Edge create(Edge edge) {
}

public List<String> create(List<Edge> edges, boolean checkVertex) {
MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.putSingle("Content-Encoding", BATCH_ENCODING);
Map<String, Object> params = ImmutableMap.of("check_vertex",
checkVertex);
RestResult result = this.client.post(this.batchPath(), edges,
headers, params);
RestHeaders headers = new RestHeaders().add(RestHeaders.CONTENT_ENCODING, BATCH_ENCODING);
Map<String, Object> params = ImmutableMap.of("check_vertex", checkVertex);
RestResult result = this.client.post(this.batchPath(), edges, headers, params);
List<String> ids = result.readList(String.class);
if (edges.size() != ids.size()) {
throw new NotAllCreatedException("Not all edges are successfully created, " +
Expand All @@ -68,10 +64,8 @@ public List<String> create(List<Edge> edges, boolean checkVertex) {

public List<Edge> update(BatchEdgeRequest request) {
this.client.checkApiVersion("0.45", "batch property update");
MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.putSingle("Content-Encoding", BATCH_ENCODING);
RestResult result = this.client.put(this.batchPath(), null,
request, headers);
RestHeaders headers = new RestHeaders().add(RestHeaders.CONTENT_ENCODING, BATCH_ENCODING);
RestResult result = this.client.put(this.batchPath(), null, request, headers);
return result.readList(this.type(), Edge.class);
}

Expand All @@ -98,11 +92,9 @@ public Edges list(int limit) {
return this.list(null, null, null, null, 0, null, limit);
}

public Edges list(Object vertexId, Direction direction,
String label, Map<String, Object> properties,
int offset, String page, int limit) {
return this.list(vertexId, direction, label, properties, false,
offset, page, limit);
public Edges list(Object vertexId, Direction direction, String label,
Map<String, Object> properties, int offset, String page, int limit) {
return this.list(vertexId, direction, label, properties, false, offset, page, limit);
}

public Edges list(Object vertexId, Direction direction, String label,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,6 @@
import java.util.Map;
import java.util.UUID;

import org.glassfish.jersey.uri.UriComponent;
import org.glassfish.jersey.uri.UriComponent.Type;

import org.apache.hugegraph.api.API;
import org.apache.hugegraph.client.RestClient;
import org.apache.hugegraph.util.E;
Expand Down Expand Up @@ -61,25 +58,14 @@ public static String formatVertexId(Object id, boolean allowNull) {
id = id.toString();
}
E.checkArgument(id instanceof String || id instanceof Number,
"The vertex id must be either String or " +
"Number, but got '%s'", id);
"The vertex id must be either String or Number, but got '%s'", id);
return (uuid ? "U" : "") + JsonUtil.toJson(id);
}

public static String formatProperties(Map<String, Object> properties) {
if (properties == null) {
return null;
}
String json = JsonUtil.toJson(properties);
/*
* Don't use UrlEncoder.encode, it encoded the space as `+`,
* which will invalidate the jersey's automatic decoding
* because it considers the space to be encoded as `%2F`
*/
return encode(json);
}

public static String encode(String raw) {
return UriComponent.encode(raw, Type.QUERY_PARAM_SPACE_ENCODED);
return JsonUtil.toJson(properties);
zhenyuT marked this conversation as resolved.
Show resolved Hide resolved
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
import org.apache.hugegraph.client.RestClient;
import org.apache.hugegraph.exception.InvalidResponseException;
import org.apache.hugegraph.exception.NotAllCreatedException;
import org.apache.hugegraph.rest.RestHeaders;
import org.apache.hugegraph.rest.RestResult;
import org.apache.hugegraph.structure.constant.HugeType;
import org.apache.hugegraph.structure.graph.BatchOlapPropertyRequest;
import org.apache.hugegraph.structure.graph.BatchVertexRequest;
Expand All @@ -32,10 +34,6 @@

import com.google.common.collect.ImmutableMap;

import jakarta.ws.rs.core.MultivaluedHashMap;

import org.apache.hugegraph.rest.RestResult;

public class VertexAPI extends GraphAPI {

public VertexAPI(RestClient client, String graph) {
Expand All @@ -53,10 +51,8 @@ public Vertex create(Vertex vertex) {
}

public List<Object> create(List<Vertex> vertices) {
MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.putSingle("Content-Encoding", BATCH_ENCODING);
RestResult result = this.client.post(this.batchPath(), vertices,
headers);
RestHeaders headers = new RestHeaders().add(RestHeaders.CONTENT_ENCODING, BATCH_ENCODING);
RestResult result = this.client.post(this.batchPath(), vertices, headers);
List<Object> ids = result.readList(Object.class);
if (vertices.size() != ids.size()) {
throw new NotAllCreatedException("Not all vertices are successfully created, " +
Expand All @@ -68,17 +64,14 @@ public List<Object> create(List<Vertex> vertices) {

public List<Vertex> update(BatchVertexRequest request) {
this.client.checkApiVersion("0.45", "batch property update");
MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.putSingle("Content-Encoding", BATCH_ENCODING);
RestResult result = this.client.put(this.batchPath(), null,
request, headers);
RestHeaders headers = new RestHeaders().add(RestHeaders.CONTENT_ENCODING, BATCH_ENCODING);
RestResult result = this.client.put(this.batchPath(), null, request, headers);
return result.readList(this.type(), Vertex.class);
}

public int update(BatchOlapPropertyRequest request) {
this.client.checkApiVersion("0.59", "olap property batch update");
MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.putSingle("Content-Encoding", BATCH_ENCODING);
RestHeaders headers = new RestHeaders().add(RestHeaders.CONTENT_ENCODING, BATCH_ENCODING);
String path = String.join("/", this.path(), "olap/batch");
RestResult result = this.client.put(path, null, request, headers);
Object size = result.readObject(Map.class).get("size");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,11 @@
import java.util.List;
import java.util.Map;

import jakarta.ws.rs.core.HttpHeaders;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.MultivaluedHashMap;

import org.apache.commons.lang3.StringUtils;

import org.apache.hugegraph.api.API;
import org.apache.hugegraph.client.RestClient;
import org.apache.hugegraph.exception.InvalidResponseException;
import org.apache.hugegraph.rest.RestHeaders;
import org.apache.hugegraph.rest.RestResult;
import org.apache.hugegraph.structure.constant.GraphMode;
import org.apache.hugegraph.structure.constant.GraphReadMode;
Expand All @@ -42,7 +38,6 @@ public class GraphsAPI extends API {
private static final String MODE = "mode";
private static final String GRAPH_READ_MODE = "graph_read_mode";
private static final String CLEAR = "clear";

private static final String CONFIRM_MESSAGE = "confirm_message";

public GraphsAPI(RestClient client) {
Expand All @@ -59,8 +54,7 @@ protected String type() {
public Map<String, String> create(String name, String cloneGraphName,
String configText) {
this.client.checkApiVersion("0.67", "dynamic graph add");
MultivaluedHashMap<String, Object> headers = new MultivaluedHashMap<>();
headers.add(HttpHeaders.CONTENT_TYPE, MediaType.TEXT_PLAIN);
RestHeaders headers = new RestHeaders().add(RestHeaders.CONTENT_TYPE, "text/plain");
Map<String, Object> params = null;
if (StringUtils.isNotEmpty(cloneGraphName)) {
params = ImmutableMap.of("clone_graph_name", cloneGraphName);
Expand Down Expand Up @@ -118,8 +112,7 @@ public void readMode(String graph, GraphReadMode readMode) {
// NOTE: Must provide id for PUT. If use "graph/graph_read_mode", "/"
// will be encoded to "%2F". So use "graph_read_mode" here although
// inaccurate.
this.client.put(joinPath(this.path(), graph, GRAPH_READ_MODE),
null, readMode);
this.client.put(joinPath(this.path(), graph, GRAPH_READ_MODE), null, readMode);
}

public GraphReadMode readMode(String graph) {
Expand Down
Loading
Loading