Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store handles in detached DDS before attaching #21132

Merged
merged 9 commits into from
May 21, 2024
26 changes: 20 additions & 6 deletions packages/runtime/datastore/src/dataStoreRuntime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -945,13 +945,27 @@ export class FluidDataStoreRuntime
// "The data store should be locally visible when generating attach summary",
// );

for (const [contextId, context] of this.contexts) {
if (!(context instanceof LocalChannelContextBase)) {
throw new LoggingError("Should only be called with local channel handles");
}
const visitedContexts = new Set<string>();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agarwal-navin: @anthony-murphy and I were wondering if this is an ok way to solve the problem of contexts for handles stored in other detached DDSs not being seen. The goal is to ensure that all contexts, even those for handles stored in a detached DDS, are able to be resolved and the handles will still get bound.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this solve the problem where binding a context results in another context binding, which the current code can miss, if the content that gets bound is before the context that binds it in the contexts list

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be good to get this in and start back porting to rc3 and rc4. we may find a better way to do this, but i think we can forward fix in main if we need to

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jzaffiro Can you explain the bug to me that you are trying to fix? The description says this is adding tests but it has code changes as well so I am not clear what is being solved here :-).

Is the problem only what Tony pointed out - "if the content that gets bound is before the context that binds it in the contexts list"? Basically, due to ordering we can miss some contexts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests are to make sure we don't hit a bug where handles that are stored in detached DDSs are not bound when we attach. The tests starting on line 501 of handleValidation.spec.ts were not able to find the DDS referenced to in the other detached DDS, which is what this code here solves. I'll update the description as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before #20998, we were also seeing a bug where going from detached --> attached would cause duplicate attach ops to be sent, and the tests starting on line 405 make sure we have coverage for all DDSs for that bug

let visitedLength = -1;
while (visitedLength !== visitedContexts.size) {
// detect changes in the visitedContexts set, as on visiting a context
// it could could make contexts available by removing other contexts
// from the notBoundedChannelContextSet, so we need to ensure those get processed as well.
// only once the loop can run with no new contexts added to the visitedContexts set do we
// know for sure all possible contexts have been visited.
visitedLength = visitedContexts.size;
for (const [contextId, context] of this.contexts) {
if (!(context instanceof LocalChannelContextBase)) {
throw new LoggingError("Should only be called with local channel handles");
}

if (!this.notBoundedChannelContextSet.has(contextId)) {
visitor(contextId, context);
if (
!visitedContexts.has(contextId) &&
!this.notBoundedChannelContextSet.has(contextId)
) {
visitor(contextId, context);
visitedContexts.add(contextId);
}
}
}
}
Expand Down
Loading
Loading