Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added CHECK_ORDER and PARALLELISM 1 to IC12 Test #1682

Merged
merged 1 commit into from
Jun 15, 2023

Conversation

rfdavid
Copy link
Collaborator

@rfdavid rfdavid commented Jun 15, 2023

What happens is: since we use multiple threads for the results, we might get different order. To solve this issue, we sort the results in the test runner.

The expected result is sorted:

[2023-06-15 12:07:41.256] [info] RESULT:

[2023-06-15 12:07:41.256] [info] 10995116278353|Otto|Muller|[Tiberius,Genghis_Khan,Justinian_I,Constantine_the_Great,Trajan]|11
[2023-06-15 12:07:41.256] [info] 13194139533500|Otto|Becker|[Tiberius,Genghis_Khan,Julius_Caesar,Alexander_the_Great,David]|5
[2023-06-15 12:07:41.256] [info] 13194139534548|Bing|Zheng|[Genghis_Khan,Hadrian,Solomon]|6
[2023-06-15 12:07:41.256] [info] 17592186044994|Jie|Wang|[Genghis_Khan,David]|7
[2023-06-15 12:07:41.256] [info] 28587302322537|Anh|Nguyen|[Mahmud_of_Ghazni,Trajan]|3
[2023-06-15 12:07:41.256] [info] 30786325578932|Alexander|Hleb|[Mahmud_of_Ghazni,David]|3
[2023-06-15 12:07:41.256] [info] 8796093022764|Zheng|Xu|[Mahmud_of_Ghazni,Ashoka,Tiberius,Marcus_Aurelius,Genghis_Khan,Justinian_I,Hadrian,Timur]|13

There are two ways to fix this:

  1. You can use sorted results in the test file if the order doesn't matter; or
  2. You can add -CHECK_ORDER to disable sorting the results and enforce using only one thread with -PARALLELISM 1
-NAME IC12
-CHECK_ORDER
-PARALLELISM 1
-QUERY MATCH ...
...

Related to #1668

@rfdavid rfdavid requested a review from aziz-mu June 15, 2023 16:17
Copy link
Contributor

@aziz-mu aziz-mu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, you can just merge if tests pass

@andyfengHKU
Copy link
Contributor

andyfengHKU commented Jun 15, 2023

What happens is: since we use multiple threads for the results, we might get different order. To solve this issue, we sort the results in the test runner.

The expected result is sorted:

[2023-06-15 12:07:41.256] [info] RESULT:

[2023-06-15 12:07:41.256] [info] 10995116278353|Otto|Muller|[Tiberius,Genghis_Khan,Justinian_I,Constantine_the_Great,Trajan]|11
[2023-06-15 12:07:41.256] [info] 13194139533500|Otto|Becker|[Tiberius,Genghis_Khan,Julius_Caesar,Alexander_the_Great,David]|5
[2023-06-15 12:07:41.256] [info] 13194139534548|Bing|Zheng|[Genghis_Khan,Hadrian,Solomon]|6
[2023-06-15 12:07:41.256] [info] 17592186044994|Jie|Wang|[Genghis_Khan,David]|7
[2023-06-15 12:07:41.256] [info] 28587302322537|Anh|Nguyen|[Mahmud_of_Ghazni,Trajan]|3
[2023-06-15 12:07:41.256] [info] 30786325578932|Alexander|Hleb|[Mahmud_of_Ghazni,David]|3
[2023-06-15 12:07:41.256] [info] 8796093022764|Zheng|Xu|[Mahmud_of_Ghazni,Ashoka,Tiberius,Marcus_Aurelius,Genghis_Khan,Justinian_I,Hadrian,Timur]|13

There are two ways to fix this:

  1. You can use sorted results in the test file if the order doesn't matter; or
  2. You can add -CHECK_ORDER to disable sorting the results and enforce using only one thread with -PARALLELISM 1
-NAME IC12
-CHECK_ORDER
-PARALLELISM 1
-QUERY MATCH ...
...

Closes #1668

Let's not close #1668 though. I think we should get the correct output on all platforms with multi-thread execution. So #1668 is still a bug that needs to be addressed in the future.

@codecov
Copy link

codecov bot commented Jun 15, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (1bd26e4) 91.16% compared to head (9b7b0a4) 91.16%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1682   +/-   ##
=======================================
  Coverage   91.16%   91.16%           
=======================================
  Files         733      733           
  Lines       26623    26623           
=======================================
  Hits        24271    24271           
  Misses       2352     2352           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@rfdavid rfdavid merged commit 054ac20 into kuzudb:master Jun 15, 2023
8 checks passed
@rfdavid rfdavid deleted the test_order_fix branch June 15, 2023 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants