-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
upload benchmark copy query times to benchmark server #2099
Conversation
Codecov ReportAll modified lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2099 +/- ##
==========================================
+ Coverage 88.91% 89.60% +0.68%
==========================================
Files 979 985 +6
Lines 36082 35750 -332
==========================================
- Hits 32082 32032 -50
+ Misses 4000 3718 -282
☔ View full report in Codecov by Sentry. |
benchmark/serializer.py
Outdated
for line in iter(process.stdout.readline, b''): | ||
print(line.decode("utf-8"), end='') | ||
if match: | ||
with open(os.path.join(benchmark_copy_log_dir, match.group(1) + '_log.txt'), 'ab') as f: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should consider move this out of the for
loop. You should not need to reopen the file each time.
benchmark/serializer.py
Outdated
process.stdin.close() | ||
match = re.match('copy (.*) from', s, re.IGNORECASE) | ||
for line in iter(process.stdout.readline, b''): | ||
print(line.decode("utf-8"), end='') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Try to switch to sys.stdout.write
and sys.stdout.flush
to see if it solves the out of order issue
Working now at: https://benchmark.kuzudb.com/run.html#2411 |
benchmark/serializer.py
Outdated
match = re.match('create\s+(.+?)\s+table\s+(.+?)\s*\(', s, re.IGNORECASE) | ||
if match: | ||
table_types[match.group(2)] = match.group(1).lower() | ||
match = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This logic is not very readable.
- You should run the check for
create
statement beforesubprocess.Popen
. If it is acreate
you should simply pass the output viasys.stdout
andsys.stderr
redirection instead of manually get the output and print. - Instead of setting
match
toFalse
here you should indent line 66-75 so it is under yourelse
. If you do this, you can also avoid theif
check for yourwrite
andflush
. - Do not reuse your flags. Call it
create_match
andcopy_match
maybe. Reusing a flag-type variable is
error-prone and make the code hard to understand.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code LGTM now. It should be merged after another round of end-to-end test.
No description provided.