Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Spark with loader has dependency conflicts #480

Merged
merged 37 commits into from
Aug 7, 2023
Merged
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
0a87143
Merge pull request #1 from apache/master
haohao0103 Sep 16, 2022
cf86be5
Merge branch 'apache:master' into master
haohao0103 Oct 21, 2022
2550ab8
Merge branch 'apache:master' into master
haohao0103 Oct 27, 2022
41e14d9
Merge branch 'apache:master' into master
haohao0103 Nov 2, 2022
6f57a3d
Merge branch 'apache:master' into master
haohao0103 Nov 4, 2022
7eca671
Merge branch 'apache:master' into master
haohao0103 Nov 7, 2022
1e7cdc8
Merge branch 'apache:master' into master
haohao0103 Nov 9, 2022
674e944
Merge branch 'apache:master' into master
haohao0103 Jan 3, 2023
6ebd8a5
Merge branch 'apache:master' into master
haohao0103 Feb 8, 2023
934cb96
Merge branch 'apache:master' into master
haohao0103 Feb 20, 2023
2eb26c7
Merge branch 'apache:master' into master
haohao0103 Apr 7, 2023
c06aa61
Merge branch 'apache:master' into master
haohao0103 May 17, 2023
ff99f1e
fix:#404
May 19, 2023
b12af0a
fix:#404
May 19, 2023
0d7e26e
Merge branch 'master' into schemaCache-optimize
simon824 May 22, 2023
60691de
Merge branch 'apache:master' into master
haohao0103 Jun 14, 2023
6993d9a
Merge branch 'apache:master' into schemaCache-optimize
haohao0103 Jun 14, 2023
bc2c8e9
fix:#464
Jun 14, 2023
de42751
fix:#464
Jun 14, 2023
7a3a0a1
fix:#464
Jun 14, 2023
eac9b18
fix:#464
Jun 14, 2023
48faf3c
fix:#464
Jun 14, 2023
d782b57
fix:#464
Jun 15, 2023
074b6ab
fix:#464
Jun 15, 2023
4e8b7ff
fix:#464
Jun 15, 2023
d51053a
fix:#464
Jun 15, 2023
a415bab
Merge branch 'apache:master' into master
haohao0103 Jun 19, 2023
ddb5e0b
Merge remote-tracking branch 'origin/master' into schemaCache-optimize
Jun 19, 2023
8f3b528
fix:#464
Jun 19, 2023
88963c1
fix:#464
Jun 19, 2023
d8b31fc
fix:#488
Jul 10, 2023
053d27b
fix:#488
Jul 11, 2023
f817163
fix:#488
Jul 12, 2023
e610fb7
Revert "fix:#488 bypass server for hbase bulk delete massive vertices…
Aug 2, 2023
b0918d5
Revert "fix:#488 bypass server for hbase bulk delete massive vertices…
Aug 2, 2023
5658a87
fix:#464
Aug 2, 2023
841e774
fix:#464
Aug 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
import org.slf4j.Logger;

import java.io.Serializable;
import java.util.Optional;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
Expand Down Expand Up @@ -292,9 +293,13 @@
case FILE:
case HDFS:
FileSource fileSource = struct.input().asFileSource();
String delimiter = fileSource.delimiter();
elements = builder.build(fileSource.header(),
row.mkString(delimiter).split(delimiter));
String delimiter = fileSource.delimiter() ;

Check warning on line 296 in hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java

View check run for this annotation

Codecov / codecov/patch

hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java#L296

Added line #L296 was not covered by tests
if (Optional.ofNullable(delimiter).isPresent()) {
elements = builder.build(fileSource.header(),
row.mkString(delimiter).split(delimiter));

Check warning on line 299 in hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java

View check run for this annotation

Codecov / codecov/patch

hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java#L298-L299

Added lines #L298 - L299 were not covered by tests
Comment on lines +297 to +299
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liuxiaocs7 could test it again?

}else {
elements = builder.build(row);

Check warning on line 301 in hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java

View check run for this annotation

Codecov / codecov/patch

hugegraph-loader/src/main/java/org/apache/hugegraph/loader/spark/HugeGraphSparkLoader.java#L301

Added line #L301 was not covered by tests
}
break;
case JDBC:
Object[] structFields = JavaConverters.asJavaCollection(row.schema().toList())
Expand Down
Loading