Specify acceptable optimizations for input roots #216
Conversation
Clarify semantics for how files in the input root should appear to actions, and how implementations may deviate from this ideal for performance reasons.
This change acts as a counter proposal to PR bazelbuild#216. Instead of globally stating whether files should be hardlinked or not, it adds the ability to let clients control whether files are aliased. It achieves this by adding a salt field to FileNode. If two files have the same salt and all other (non-name) properties match, they SHOULD get constructed in the form of hardlinks.
|
Related: bazelbuild/bazel#10299 is a Bazel issue that requests the ability to mark files as non-symlinkable, to be more compatible with JavaScript ecosystem tools. The workaround I used in that case was inelegant: forcing non-symlink placement for any files that are executable or end in It would be nice if clients could tag specific files as needing special treatment, even if that tag were as general-purpose as "do not optimize placement or storage of this file at all". |
This change acts as a counter proposal to PR bazelbuild#216. Instead of globally stating whether files should be hardlinked or not, it adds the ability to let clients control whether files are aliased. It achieves this by adding a salt field to FileNode. If two files have the same salt and all other (non-name) properties match, they SHOULD get constructed in the form of hardlinks.
This change acts as a counter proposal to PR bazelbuild#216. Instead of globally stating whether files should be hardlinked or not, it adds the ability to let clients control whether files are aliased. It achieves this by adding a salt field to FileNode. If two files have the same salt and all other (non-name) properties match, they SHOULD get constructed in the form of hardlinks.
|
It might make sense to have more of a discussion around what the options and expectations are:
I think most people would expect that the files are standalone (copy or clonefile), and that they can be modified. However, in our experience, copy is the slowest option, and hardlinked files have to be read-only to avoid corruption. We have an undocumented platform option to select between these options as an escape hatch (can be set on a per-action basis), but haven't actually needed it so far. |
Clarify semantics for how files in the input root should appear to actions, and how implementations may deviate from this ideal for performance reasons.
This wording is proposed as one possible direction we could specify, for further discussion following conversation at the Remote Execution WG monthly.
@EdSchouten please go ahead and propose your preferred alternative. This is not intended to be committed until/unless we've had a chance to see the alternatives and attain consensus here.