-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remote backend for PyG #1418
Remote backend for PyG #1418
Conversation
7caff52
to
fbd7071
Compare
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #1418 +/- ##
==========================================
+ Coverage 92.64% 92.72% +0.07%
==========================================
Files 650 651 +1
Lines 22613 22749 +136
==========================================
+ Hits 20949 21093 +144
+ Misses 1664 1656 -8
... and 1 file with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
52bd939
to
8c098f6
Compare
src/include/main/storage_driver.h
Outdated
|
||
~StorageDriver(); | ||
|
||
std::pair<std::unique_ptr<uint8_t[]>, size_t> scan(const std::string& nodeName, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would wrap the pair into a struct:
struct Buffer {
std::unique_ptr<uint8_t> data;
std::size_t size;
}
auto frame = bufferManager.pin(*fileHandleToPin, pageIdxToPin); | ||
auto frameBytesOffset = getElemByteOffset(cursor.elemPosInPage); | ||
memcpy(result + i * elementSize, frame + frameBytesOffset, elementSize); | ||
bufferManager.unpin(*fileHandleToPin, pageIdxToPin); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
L19 - L22 :
bufferManager.optimisticRead(*fileHandleToPin, pageIdxToPin, [&](uint8_t* frame) -> void {
auto frameBytesOffset = getElemByteOffset(cursor.elemPosInPage);
memcpy(result + i * elementSize, frame + frameBytesOffset, elementSize);
});
0f22317
to
19b5e0a
Compare
19b5e0a
to
6ee6705
Compare
std::unordered_map<std::string, std::shared_ptr<Value>> parameters; | ||
parameters["s"] = std::make_shared<Value>(start); | ||
parameters["e"] = std::make_shared<Value>(end); | ||
std::cout << "Batch " << batch << " from " << start << " to " << end << std::endl; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should remove before the final release. But let's keep it for benchmark purpose
No description provided.