Rebase to v2.46.1 #686
Merged
Rebase to v2.46.1 #686
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Start work on a new `git survey` command to scan the repository for monorepo performance and scaling problems. The goal is to measure the various known "dimensions of scale" and serve as a foundation for adding additional measurements as we learn more about Git monorepo scaling problems. Results will be logged to the console and to Trace2. The initial goal is to complement the scanning and analysis performed by the GO-based `git-sizer` (https://github.com/github/git-sizer) tool. It is hoped that by creating a builtin command, we may be able to take advantage of internal Git data structures and code that is not accessible from GO to gain further insight into potential scaling problems. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
By default we will scan all references in "refs/heads/", "refs/tags/" and "refs/remotes/". Add command line opts let the use ask for all refs or a subset of them and to include a detached HEAD. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Collect the set of requested branches, tags, and etc into a ref_array and collect the set of requested patterns into a strvec. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
In ac8acb4 (sparse-index: complete partial expansion, 2022-05-23), 'expand_index()' was updated to expand the index to a given pathspec. However, the 'path_matches_pattern_list()' method used to facilitate this has the side effect of initializing or updating the index hash variables ('name_hash', 'dir_hash', and 'name_hash_initialized'). This operation is performed on 'istate', though, not 'full'; as a result, the initialized hashes are later overwritten when copied from 'full'. To ensure the correct hashes are in 'istate' after the index expansion, change the arg used in 'path_matches_pattern_list()' from 'istate' to 'full'. Note that this does not fully solve the problem. If 'istate' does not have an initialized 'name_hash' when its contents are copied to 'full', initialized hashes will be copied back into 'istate' but 'name_hash_initialized' will be 0. Therefore, we also need to copy 'full->name_hash_initialized' back to 'istate' after the index expansion is complete. Signed-off-by: Victoria Dye <vdye@github.com>
Add test case to demonstrate that `git index-pack -o <idx-path> pack-path` fails if <idx-path> does not end in ".idx" when `--rev-index` is enabled. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. The test here demonstrates the failure. (The test forces `--rev-index` to avoid interaction with `GIT_TEST_NO_WRITE_REV_INDEX` during CI runs.) Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Calculate stats on the set of refs. This includes the number of branches, plain and annotated tags, remotes, and etc. Calculate the number of packed vs loose refs. Calculate the size of the set of refnames. Print results on the console in JSON format. Add Trace2 logging of the results as a data_json event. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
These seem to be custom tests to microsoft/git as they break without these changes, but these changes are not needed upstream. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Add a test verifying that sparse-checkout (with and without sparse index enabled) treat untracked files & directories correctly when changing sparse patterns. Specifically, it ensures that 'git sparse-checkout set' * deletes empty directories outside the sparse cone * does _not_ delete untracked files outside the sparse cone Signed-off-by: Victoria Dye <vdye@github.com>
Teach index-pack to silently omit the reverse index if the index file does not have the standard ".idx" suffix. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. Since we do not know why the user requested a non-standard suffix for the index, we cannot guess what the proper corresponding suffix should be for the reverse index. So we disable it. The t5300 test has been updated to verify that we no longer error out and that the .rev file is not created. TODO We could warn the user that we skipped it (perhaps only if they TODO explicitly requested `--rev-index` on the command line). TODO TODO Ideally, we should add an `--rev-index-path=<path>` argument TODO or change `--rev-index` to take a pathname. TODO TODO I'll leave these questions for a future series. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Prefetch the value of GIT_TRACE2_DST_DEBUG during startup and before we try to open any Trace2 destination pathnames. Normally, Trace2 always silently fails if a destination target cannot be opened so that it doesn't affect the execution of a Git command. The command should run normally, but just not generate any trace data. This can make it difficult to debug a telemetry setup, since the user doesn't know why telemetry isn't being generated. If the environment variable GIT_TRACE2_DST_DEBUG is true, the Trace2 startup will print a warning message with the `errno` to make debugging easier. However, on Windows, looking up the env variable resets `errno` so the warning message always ends with `...tracing: No error` which is not very helpful. Prefetch the env variable at startup. This avoids the need to update each call-site to capture `errno` in the usual `saved-errno` variable. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Add treewalk on the commits and objects reachable from the set of refs. This commit sets up the treewalk, but only stubs in the traverse callbacks. We'll actually look at the commit and object data in the next commit. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Add callback to handle commit objects during the treewalk. Count the number of commits and group them by the number of parents. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Create `struct large_item` and `struct large_item_vec` to capture the n largest commits, trees, and blobs under various scaling dimensions, such as size in bytes, number of commit parents, or number of entries in a tree. Each of these have a command line option to set them independently. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Include the pathname of each blob or tree in the large_item_vec to help identify the file or directory associated with the OID and size information. This pathname is computed during the treewalk, so it reflects the first observed pathname seen for that OID during the traversal over all of the refs. Since the file or directory could have moved (without being modified), there may be multiple "correct" pathnames for a particular OID. Since we do not control the ref traversal order, we should consider it to be a "suggested pathname" for the OID. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Computing `git name-rev` on each commit, tree, and blob in each of the various large_item_vec can be very expensive if there are too many refs, especially if the user doesn't need the result. Lets make it optional. The `--no-name-rev` option can save 50 calls to `git name-rev` since we have 5 large_item_vec's and each defaults to 10 items. Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
This topic branch brings in a new, experimental built-in command to assess the dimensions of a local repository. It is experimental and subject to change! It might grow new options, change its output, or even be moved into `git diagnose --analyze` or something like that. The hope is that this command, which was inspired by `git sizer` (https://github.com/github/git-sizer), will be helpful not only in diagnosing issues with large repositories, but also in modeling what shapes and sizes of repositories can be handled by Git (and as a corollary: where Git needs to improve to be able to accommodate the natural growth of repositories). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This backports the `ds/advice-sparse-index-expansion` patches into `microsoft/git` which _just_ missed the v2.46.0 window. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
While this command is definitely something we _want_, chances are that upstreaming this will require substantial changes. We still want to be able to experiment with this before that, to focus on what we need out of this command: To assist with diagnosing issues with large repositories, as well as to help monitoring the growth and the associated painpoints of such repositories. To that end, we are about to integrate this command into `microsoft/git`, to get the tool into the hands of users who need it most, with the idea to iterate in close collaboration between these users and the developers familar with Git's internals. However, we will definitely want to avoid letting anybody have the impression that this command, its exact inner workings, as well as its output format, are anywhere close to stable. To make that fact utterly clear (and thereby protect the freedom to iterate and innovate freely before upstreaming the command), let's mark its output as experimental in all-caps, as the first thing we do. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
While using the reset --stdin feature on windows path added may have a \r at the end of the path that wasn't getting removed so didn't match the path in the index and wasn't reset. Signed-off-by: Kevin Willford <kewillf@microsoft.com>
It has been a long-standing practice in Git for Windows to append `.windows.<n>`, and in microsoft/git to append `.vfs.0.0`. Let's keep doing that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since we really want to be based on a `.vfs.*` tag, let's make sure that there was a new-enough one, i.e. one that agrees with the first three version numbers of the recorded default version. This prevents e.g. v2.22.0.vfs.0.<some-huge-number>.<commit> from being used when the current release train was not yet tagged. It is important to get the first three numbers of the version right because e.g. Scalar makes decisions depending on those (such as assuming that the `git maintenance` built-in is not available, even though it actually _is_ available). Signed-off-by: Johannes Schindelin <johasc@microsoft.com>
The diff_populate_filespec() method is used to describe the diff after a merge operation is complete, especially when a conflict appears. In order to avoid expanding a sparse index, the reuse_worktree_file() needs to be adapted to ignore files that are outside of the sparse-checkout cone. The file names and OIDs used for this check come from the merged tree in the case of the ORT strategy, not the index, hence the ability to look into these paths without having already expanded the index. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
…e after clones These are two highly-requested items from an internal team considering a move to Scalar using Azure Repos. 1. Remove the requirement that we create a `src` directory at clone time. 2. Allow `git worktree` even when using the GVFS protocol. These are not difficult to implement. The `--no-src` option could even be submitted upstream (though the commit will need to drop one bit about an interaction with the local cache path).
Upstream, a20f704 (add: warn when asked to update SKIP_WORKTREE entries, 04-08-2021) modified how 'git add <pathspec>' works with cache entries marked with the SKIP_WORKTREE bit. The intention is to prevent a user from accidentally adding a path that is outside their sparse-checkout definition but somehow matches an existing index entry. This breaks when using the virtual filesystem in VFS for Git. It is rare, but we could be in a scenario where the user has staged a change and then the file is projected away. If the user re-adds the file, then this warning causes the command to fail with the advise message. Disable this logic when core_virtualfilesystem is enabled. This should allow the VFS for Git functional tests to pass (at least the ones in the default run). I'll create a `-pr` installer build to check before merging this.
There is some strangeness when expanding a sparse-index that exists within a submodule. We will need to resolve that later, but for now, let's do a better job of explicitly disabling the sparse-index when requested, and do so in t7817. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
``` 6e74958 p2000: add 'git checkout -' test and decrease depth 3e1d03c p2000: compress repo names cd94f82 commit: integrate with sparse-index 65e79b8 sparse-index: recompute cache-tree e9a9981 checkout: stop expanding sparse indexes 4b801c8 t1092: document bad 'git checkout' behavior 71e3015 unpack-trees: resolve sparse-directory/file conflicts 5e96df4 t1092: test merge conflicts outside cone defab1b add: allow operating on a sparse-only index 9fc4313 pathspec: stop calling ensure_full_index 0ec03ab add: ignore outside the sparse-checkout in refresh() adf5b15 add: remove ensure_full_index() with --renormalize ``` These commits are equivalent to those already in `next` via gitgitgadget#999. ``` 80b8d6c Merge branch 'sparse-index/add' into stolee/sparse-index/add ``` This merge resolves conflicts with some work that happened in parallel, but is already in upstream `master`. ``` c407b2c t7519: rewrite sparse index test 9dad0d2 sparse-index: silently return when not using cone-mode patterns 2974920 sparse-index: silently return when cache tree fails e7cdaa0 unpack-trees: fix nested sparse-dir search 347410c sparse-checkout: create helper methods 4537233 attr: be careful about sparse directories 5282a86 sparse-index: add SPARSE_INDEX_MEMORY_ONLY flag 3a2f316 sparse-checkout: clear tracked sparse dirs fb47b56 sparse-checkout: add config to disable deleting dirs ``` These commits are the ones under review as of gitgitgadget#1009. Recent review made this less stable. It's a slightly different and more robust version of #396. > Note: I'm still not done with the feedback for upstream, but the remaining feedback is "can we add tests that cover these tricky technical bits?" and in `microsoft/git` these are already covered by the Scalar functional tests (since that's how they were found). ``` 080b02c diff: ignore sparse paths in diffstat d91a647 merge: make sparse-aware with ORT df49b5f merge-ort: expand only for out-of-cone conflicts cdecb85 t1092: add cherry-pick, rebase tests 0c1ecfb sequencer: ensure full index if not ORT strategy 406dfbe sparse-index: integrate with cherry-pick and rebase ``` These commits integrate with `git merge`, `git cherry-pick`, `git revert`, and `git rebase` as of gitgitgadget#1019. This got some feedback that changed how the tests were working so they are more robust. This led to a new commit (0c1ecfb). ``` cbb0ab3 Merge branch 'sparse-index/merge' into vfs-2.33.0 acb8623 t7524: test no longer fails ``` Finally, the commits are merged into `vfs-2.33.0` and also we include a fix to a `microsoft/git` test that is no longer broken.
This verifies that `diff` and `diff --staged` behave the same in sparse
index repositories in the following partially-staged scenarios (i.e. the
index, HEAD, and working directory differ at a given path):
1. Path is within sparse-checkout cone.
2. Path is outside sparse-checkout cone.
3. A merge conflict exists for paths outside sparse-checkout cone.
Signed-off-by: Lessley Dennington <lessleydennington@gmail.com>
This branch is exactly #410, but with one more commit: enabling the sparse index by default in d59110a. Having this in the `vfs-2.33.0` branch helps build confidence that the sparse index is doing what it should be doing by running in the Scalar functional tests and in our test branches. If we want to cut a new `microsoft/git` release without enabling the sparse index, we can simply revert this commit.
Test cases specific to handling untracked files in `git stash` a) ensure that files outside the sparse checkout definition are handled as-expected and b) document the index expansion inside of `git stash -u`. Note that, in b), it is not the full repository index that is expanded - it is the temporary, standalone index containing the stashed untracked files only. Signed-off-by: Victoria Dye <vdye@github.com>
diff: enable and test the sparse index
Sparse index: integrate with `clean` and `stash -u`
…irected During a run of the Scalar functional tests, we hit a case where the inexact rename detection of a 'git cherry-pick' command slowed to the point of writing its delayed progress, failing the test because stderr differed from the control case. Showing progress like this when stderr is not a terminal is non-standard for Git, so inject an isatty(2) when initializing the progress option in sequencer.c. Unfortunately, there is no '--quiet' option in 'git cherry-pick' currently wired up. This could be considered in the future, and the isatty(2) could be moved to that position. This would also be needed for commands like 'git rebase', so we leave that for another time.
During the 2.35.0 rebase, we ejected 570f64b (Fix reset when using the sparse-checkout feature., 2017-03-15) because of a similar change upstream that actually works with the expected behavior of sparse-checkout. That commit only ever existed in microsoft/git, but when it was considered for upstream we realized that it behaved strangely for a sparse-checkout scenario. The root problem is that during a mixed reset, 'git reset <commit>' updates the index to aggree with <commit> but leaves the worktree the same as it was before. The issue with sparse-checkout is that some files might not be in the worktree and thus the information from those files would be "lost". The upstream decision was to leave these files as ignored, because that's what the SKIP_WORKTREE bit means: don't put these files in the worktree and ignore their contents. If there already were files in the worktree, then Git does not change them. The case for "losing" data is if a committed change outside of the sparse-checkout was in the previous HEAD position. However, this information could be recovered from the reflog. The case where this is different is in a virtualized filesystem. The virtualization is projecting the index contents onto the filesystem, so we need to do something different here. In a virtual environment, every file is considered "important" and we abuse the SKIP_WORKTREE bit to indicate that Git does not need to process a projected file. When a file is populated, the virtual filesystem hook provides the information for removing the SKIP_WORKTREE bit. In the case of these mixed resets, we have the issue where we change the projection of the worktree for these cache entries that change. If a file is populated in the worktree, then the populated file will persist and appear in a follow-up 'git status'. However, if the file is not populated and only projected, we change the projection from the current value to the new value, leaving a clean 'git status'. The previous version of this commit includes a call to checkout_entry(), which populates the file. This causes the file to be actually in the working tree and no longer projected. To make this work with the upstream changes, stop setting the skip-worktree bit for the new cache entry. This seemed to work fine without this change, but it's likely due to some indirection with the virtual filesystem. Better to do the best-possible thing here so we don't hide a corner-case bug by accident. Helped-by: Victoria Dye <vdye@github.com> Signed-off-by: Kevin Willford <kewillf@microsoft.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com>
* t1092: remove the 'git update-index' test that currently fails because the command ignores the bad path, but doesn't return a failure. * dir.c: prevent matching against sparse-checkout patterns when the virtual filesystem is enabled. Should prevent some corner case issues. * t1092: add quiet mode for some rebase tests because the stderr output can change in some of the modes. Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
…lesystem This replaces #493 (can't reopen a PR after a force-push...). I updated this commit with a more firm version of the fix. This hopefully answers Victoria's excellent concerns with the previous approach. I did not manage to get an automated test for this, but I did carefully verify this manually with a few commits in a VFS for Git enlistment (with different files every time). I updated the commit message with more details about why this works. --- This fork contains changes specific to monorepo scenarios. If you are an external contributor, then please detail your reason for submitting to this fork: * [X] This change only applies to the virtualization hook and VFS for Git. Resolves #490.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
…x-built-in-fsmonitor Fix the built-in FSMonitor, and run Scalar's Functional Tests as part of the automated builds
This is random stuff that probably all got upstream in the meantime.
mjcheetham
approved these changes
Sep 18, 2024
|
For the record: I am convinced that this build failure is due to cURL v8.10.0 having a regression, but v8.10.1 was released this morning (and obviously did not arrive at that runner's Homebrew installation yet): It is not something that can be fixed by Git, and therefore I will ignore it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Range-diff relative to vfs-2.46.0
7: ec811a0 = 1: 0082dc7 survey: stub in new experimental
git-surveycommand8: 93e532c = 2: 5a54e19 survey: add command line opts to select references
9: 80b37d2 = 3: ca6d3d8 survey: collect the set of requested refs
2: 5ee9c43 = 4: 9c19694 sparse-index.c: fix use of index hashes in expand_index
4: 8530990 = 5: 297189c t5300: confirm failure of git index-pack when non-idx suffix requested
10: a9edb5a = 6: 4c9a2e2 survey: calculate stats on refs and print results
1: d2d4035 = 7: 3c7548c t: remove advice from some tests
3: ad14bf4 = 8: 0ffd36c t1092: add test for untracked files and directories
5: abc8681 ! 9: 22f5216 index-pack: disable rev-index if index file has non .idx suffix
6: 50c7799 = 10: d361c6d trace2: prefetch value of GIT_TRACE2_DST_DEBUG at startup
11: cc84ca6 = 11: 62d0c8c survey: stub in treewalk of reachable commits and objects
12: 8276f9b = 12: b7da696 survey: add traverse callback for commits
13: 9514425 = 13: e388414 survey: add vector of largest objects for various scaling dimensions
14: 57d85c3 = 14: 7905a6e survey: add pathname of blob or tree to large_item_vec
15: 753a436 = 15: dd0c573 survey: add commit-oid to large_item detail
16: 145b957 = 16: 6c6b28c survey: add commit name-rev lookup to each large_item
17: bb75fa6 = 17: 5a6a80b survey: add --json option and setup for pretty output
18: f3098cb = 18: c84296d survey: add pretty printing of stats
19: a5febc9 = 19: 592b09f t8100: create test for git-survey
20: f5d1032 = 20: 494e9c5 survey: add --no-name-rev option
21: 923582c = 21: 2332438 survey: started TODO list at bottom of source file
22: 6ad83b7 = 22: 4b9fd55 survey: expanded TODO list at the bottom of the source file
23: b9e50f7 = 23: 95bc929 survey: expanded TODO with more notes
24: 891f502 = 24: bd44fa9 survey: clearly note the experimental nature in the output
25: 97af855 = 25: d5ff2d2 reset --stdin: trim carriage return from the paths
26: 250cbe1 ! 26: 0f6a2ed Identify microsoft/git via a distinct version suffix
27: a7f73ee = 27: 81e5a4c gvfs: ensure that the version is based on a GVFS tag
28: c165478 = 28: a441c1f gvfs: add a GVFS-specific header file
29: cea7ec7 = 29: 4cfd73a gvfs: add the core.gvfs config setting
30: 3648554 = 30: d94ca7d gvfs: add the feature to skip writing the index' SHA-1
31: 08b42f2 = 31: 363f882 gvfs: add the feature that blobs may be missing
32: 86ee5ec = 32: 4bd9426 gvfs: prevent files to be deleted outside the sparse checkout
33: d0b5dcf = 33: 77d8f4a gvfs: optionally skip reachability checks/upload pack during fetch
34: fcc15b3 = 34: 4c1c45a gvfs: ensure all filters and EOL conversions are blocked
35: c6dc4e5 = 35: 695e263 gvfs: allow "virtualizing" objects
36: a0db91f = 36: 5bcdb2c Hydrate missing loose objects in check_and_freshen()
37: b9e1fc6 = 37: b1fd1df sha1_file: when writing objects, skip the read_object_hook
38: 6805073 = 38: a168919 gvfs: add global command pre and post hook procs
39: 18705c3 = 39: 8e7be74 t0400: verify that the hook is called correctly from a subdirectory
40: 63eab82 = 40: f27a1ff Pass PID of git process to hooks.
41: 922a8cf = 41: a64248f pre-command: always respect core.hooksPath
42: 78b96ea = 42: 7a255d4 sparse-checkout: update files with a modify/delete conflict
43: 98f7773 = 43: 8417502 sparse-checkout: avoid writing entries with the skip-worktree bit
44: e5d4c1d = 44: 7be6acd Do not remove files outside the sparse-checkout
45: 9f1b23e = 45: 783668a send-pack: do not check for sha1 file when GVFS_MISSING_OK set
46: 48ba4c2 = 46: deee997 cache-tree: remove use of strbuf_addf in update_one
47: c15e53e = 47: 335dfe0 gvfs: block unsupported commands when running in a GVFS repo
48: e90bfb5 = 48: 6a34107 worktree: allow in Scalar repositories
49: 8400e33 = 49: fe61f9e gvfs: allow overriding core.gvfs
50: 233bf52 = 50: 234684e BRANCHES.md: Add explanation of branches and using forks
51: af81bfc = 51: a833057 Add virtual file system settings and hook proc
52: 905318f = 52: 487969d virtualfilesystem: don't run the virtual file system hook if the index has been redirected
53: 090c57e = 53: 4ddc91e virtualfilesystem: check if directory is included
54: fb00b66 = 54: 9d2dedb backwards-compatibility: support the post-indexchanged hook
55: bbff50a = 55: 5f83dae gvfs: verify that the built-in FSMonitor is disabled
56: f7476e9 = 56: ba33c06 wt-status: add trace2 data for sparse-checkout percentage
57: be104c6 = 57: 9995935 wt-status: add VFS hydration percentage to normal
git statusoutput58: 520f402 = 58: 5e9b797 status: add status serialization mechanism
59: 790b937 = 59: 601706a Teach ahead-behind and serialized status to play nicely together
60: cdd2535 = 60: 510ca56 status: serialize to path
61: 453d800 = 61: 5ef9e22 status: reject deserialize in V2 and conflicts
62: a9aa37c = 62: 1076a66 serialize-status: serialize global and repo-local exclude file metadata
63: 29d7746 = 63: 27e5fa9 status: deserialization wait
64: 87570db = 64: c287796 merge-recursive: avoid confusing logic in was_dirty()
65: 2872f85 = 65: 757f5d7 merge-recursive: add some defensive coding to was_dirty()
66: d53d1ae = 66: c295584 merge-recursive: teach was_dirty() about the virtualfilesystem
67: 6b02fcd = 67: c766116 status: deserialize with -uno does not print correct hint
68: 775d881 = 68: d9b2bce fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate
69: 1956956 = 69: a92b304 fsmonitor: add script for debugging and update script for tests
70: 3284d55 = 70: 418ce15 status: disable deserialize when verbose output requested.
71: 2103d34 = 71: 0c260d9 t7524: add test for verbose status deserialzation
72: 9a4b9e9 = 72: 52dc6c6 deserialize-status: silently fallback if we cannot read cache file
73: e665d87 = 73: 863ebb5 gvfs:trace2:data: add trace2 tracing around read_object_process
74: 4d814a7 = 74: 865bb62 gvfs:trace2:data: status deserialization information
75: f50b605 = 75: a7f1150 gvfs:trace2:data: status serialization
76: 5869b3f = 76: 237b0dd gvfs:trace2:data: add vfs stats
77: a3e8249 = 77: 67070df trace2: refactor setting process starting time
78: 31624c4 = 78: 2e027f7 trace2:gvfs:experiment: clear_ce_flags_1
79: 33bae12 = 79: e9d77c0 trace2:gvfs:experiment: report_tracking
80: 8b55ade = 80: a066334 trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache
81: 4ee89cd ! 81: 3cd3022 trace2:gvfs:experiment: read-cache: time read/write of cache-tree extension
@@ read-cache.c: static int do_write_index(struct index_state *istate, struct tempf + trace2_region_leave("index", "write/extension/cache_tree", NULL); + strbuf_release(&sb); - if (err) - return -1; + /* + * NEEDSWORK: write_index_ext_header() never returns a failure,82: bec8264 = 82: a6dbfe4 trace2:gvfs:experiment: add region to apply_virtualfilesystem()
83: d019329 = 83: 619b945 trace2:gvfs:experiment: add region around unpack_trees()
84: e035360 = 84: e9ce59e trace2:gvfs:experiment: add region to cache_tree_fully_valid()
85: 571fd76 = 85: 69cf26e trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() and report_tracking()
86: f2c0f33 = 86: d5d0685 trace2:gvfs:experiment: increase default event depth for unpack-tree data
87: 9773e2e = 87: bcbacc0 trace2:gvfs:experiment: add data for check_updates() in unpack_trees()
88: 64b58c0 = 88: ae9ea80 Trace2:gvfs:experiment: capture more 'tracking' details
89: 8212ec4 = 89: 82bf4da credential: set trace2_child_class for credential manager children
90: 6a0cbb3 = 90: 3f3b8c0 sub-process: do not borrow cmd pointer from caller
91: 8da9f58 = 91: bf56ae8 sub-process: add subprocess_start_argv()
92: e0dd3e1 = 92: 9af167d sha1-file: add function to update existing loose object cache
93: ddd1116 = 93: b576d35 packfile: add install_packed_git_and_mru()
94: cf33e1a = 94: d004438 index-pack: avoid immediate object fetch while parsing packfile
95: 0f74b95 = 95: 603ac02 gvfs-helper: create tool to fetch objects using the GVFS Protocol
96: 0ee58cc = 96: 9eb624f sha1-file: create shared-cache directory if it doesn't exist
97: 4a7deea = 97: 446fe4c gvfs-helper: better handling of network errors
98: 85150e7 = 98: 348147c gvfs-helper-client: properly update loose cache with fetched OID
99: 6af3a60 = 99: bcab9a9 gvfs-helper: V2 robust retry and throttling
100: 973905f = 100: 7b6ee22 gvfs-helper: expose gvfs/objects GET and POST semantics
101: b247583 = 101: 9f0699f gvfs-helper: dramatically reduce progress noise
102: 423e53d = 102: deefbdc gvfs-helper-client.h: define struct object_id
103: 69baf81 = 103: c19200a gvfs-helper: handle pack-file after single POST request
104: 944e394 = 104: cd7b1b8 test-gvfs-prococol, t5799: tests for gvfs-helper
105: fdb8ece = 105: 6bb3ef8 gvfs-helper: move result-list construction into install functions
106: 0b236aa = 106: 68080f7 t5799: add support for POST to return either a loose object or packfile
107: 0edf612 = 107: 9cf1ba6 t5799: cleanup wc-l and grep-c lines
108: db3f4c3 = 108: f21f2d5 gvfs-helper: verify loose objects after write
109: 1bf5739 = 109: f49f07d t7599: create corrupt blob test
110: 90aa80c = 110: 20657aa gvfs-helper: add prefetch support
111: 8dbc977 = 111: 73409ad gvfs-helper: add prefetch .keep file for last packfile
112: db28521 = 112: 6f6779a gvfs-helper: do one read in my_copy_fd_len_tail()
113: d1127cc = 113: afe9833 gvfs-helper: move content-type warning for prefetch packs
114: 5077a1d = 114: a7ba4d6 fetch: use gvfs-helper prefetch under config
115: cae9757 = 115: 514d5d3 gvfs-helper: better support for concurrent packfile fetches
116: a7f52d2 = 116: a0c575e remote-curl: do not call fetch-pack when using gvfs-helper
117: 8661bc3 = 117: 0a766d8 fetch: reprepare packs before checking connectivity
118: aa0f581 = 118: c03ec2f gvfs-helper: retry when creating temp files
119: 4c4e6d3 = 119: 653d77d sparse: avoid warnings about known cURL issues in gvfs-helper.c
120: 0818412 = 120: 1761a30 gvfs-helper: add --max-retries to prefetch verb
121: 0897986 = 121: dd843bb t5799: add tests to detect corrupt pack/idx files in prefetch
122: 3fecda7 = 122: 92da944 gvfs-helper: ignore .idx files in prefetch multi-part responses
123: 3e8a7c4 = 123: 3234bdc t5799: explicitly test gvfs-helper --fallback and --no-fallback
124: c5d5aba = 124: de72a93 gvfs-helper: don't fallback with new config
127: 27ad5ea = 125: 839d493 maintenance: care about gvfs.sharedCache config
125: a1af705 = 126: 3663e7e test-gvfs-protocol: add cache_http_503 to mayhem
128: cd4a4d3 = 127: 9a6377d unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags
126: 32e0842 = 128: e771d82 t5799: add unit tests for new
gvfs.fallbackconfig setting129: b7083b7 = 129: 101945b homebrew: add GitHub workflow to release Cask
130: 90e57bc = 130: 77e27ca Adding winget workflows
131: b6fb09b = 131: bbb2ad5 Disable the
monitor-componentsworkflow in msft-git132: ad407da = 132: 3e25db3 .github: enable windows builds on microsoft fork
133: 7ff9577 ! 133: 0b19ffc release: create initial Windows installer build workflow
134: 23fb420 = 134: ee7951a help: special-case HOST_CPU
universal135: be4dd9a ! 135: c668ede release: add Mac OSX installer build
136: f0cfe11 ! 136: ee6ac30 release: build unsigned Ubuntu .deb package
137: 6d3ddc3 ! 137: 845e9d6 release: add signing step for .deb package
@@ .github/workflows/build-git-installers.yml: jobs: + debsigs --sign=origin --verify --check microsoft-git_"$version".deb + - name: Upload artifacts - uses: actions/upload-artifact@v3 + uses: actions/upload-artifact@v4 with: name: linux-artifacts path: |138: 245f212 ! 138: 12b573b release: create draft GitHub release with packages & installers
139: 68f3e0e = 139: cb20661 build-git-installers: publish gpg public key
140: 365a742 = 140: 400e242 release: continue pestering until user upgrades
141: df589f7 = 141: 66ca679 Makefile: allow specifying GIT_BUILT_FROM_COMMIT
142: 6b3ac94 = 142: a168637 dist: archive HEAD instead of HEAD^{tree}
143: 05280bd = 143: 64a8621 release: include GIT_BUILT_FROM_COMMIT in MacOS build
145: e21f5f8 = 144: 22f5bc6 update-microsoft-git: create barebones builtin
144: 608de82 ! 145: 9de58d0 release: add installer validation
146: 23bc4b5 = 146: f759434 update-microsoft-git: Windows implementation
151: 19f75e7 = 147: 6bb3782 git_config_set_multivar_in_file_gently(): add a lock timeout
147: 35ea029 = 148: 4897e1f update-microsoft-git: use brew on macOS
152: f9a9bae = 149: 6816304 scalar: set the config write-lock timeout to 150ms
148: ab00c5a = 150: a7390fa .github: update ISSUE_TEMPLATE.md for microsoft/git
153: 17b8d55 = 151: 9aecedb scalar: add docs from microsoft/scalar
149: a12bc1a = 152: 7372c14 .github: update PULL_REQUEST_TEMPLATE.md
154: c3dbc93 = 153: 4d7b52c scalar (Windows): use forward slashes as directory separators
155: 003d4cb = 154: 8f56aa9 scalar: add retry logic to run_git()
156: cb28a5c = 155: d38da58 scalar: support the
configcommand for backwards compatibility176: 68982db = 156: 942d84a credential: add new interactive config option
177: 4b2c98f = 157: 40ac109 maintenance: add custom config to background jobs
187: f90edc6 = 158: 0d6a051 sequencer: avoid progress when stderr is redirected
150: 7a74adb ! 159: 3391dff Adjust README.md for microsoft/git
@@ Commit message Microsoft's fork of Git is not quite Git for Windows, therefore we want to tell the keen reader all about it. :-) + Co-authored-by: Kyle Rader <kyrader@microsoft.com> + Co-authored-by: Victoria Dye <vdye@github.com> + Co-authored-by: Jeff Hostetler <jeffhost@microsoft.com> + Co-authored-by: Matthew Cheetham <mattche@microsoft.com> + Co-authored-by: Derrick Stolee <dstolee@microsoft.com> + Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> + Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> ## README.md ## @@ @@ README.md +[GitHub issues](https://github.com/microsoft/git/issues). + +Why is this fork needed? ++========================================================= ++ ++Git is awesome - it's a fast, scalable, distributed version control system with an unusually rich ++command set that provides both high-level operations and full access to internals. What more could ++you ask for? ++ ++Well, because Git is a distributed version control system, each Git repository has a copy of all ++files in the entire history. As large repositories, aka _monorepos_ grow, Git can struggle to ++manage all that data. As Git commands like `status` and `fetch` get slower, developers stop waiting ++and start switching context. And context switches harm developer productivity. ++ ++`microsoft/git` is focused on addressing these performance woes and making the monorepo developer ++experience first-class. The Scalar CLI packages all of these recommendations into a simple set of ++commands. ++ ++One major feature that Scalar recommends is [partial clone](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/), ++which reduces the amount of data transferred in order to work with a Git repository. While several ++services such as GitHub support partial clone, Azure Repos instead has an older version of this ++functionality called ++[the GVFS protocol](https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md). ++The integration with the GVFS protocol present in `microsoft/git` is not appropriate to include in ++the core Git client because partial clone is the official version of that functionality. ++ ++Downloading and Installing ========================================================= -Git is a fast, scalable, distributed revision control system with an @@ README.md -[Documentation/SubmittingPatches]: Documentation/SubmittingPatches -[Documentation/CodingGuidelines]: Documentation/CodingGuidelines -[po/README.md]: po/README.md -+Git is awesome - it's a fast, scalable, distributed version control system with an unusually rich -+command set that provides both high-level operations and full access to internals. What more could -+you ask for? -+ -+Well, because Git is a distributed version control system, each Git repository has a copy of all -+files in the entire history. As large repositories, aka _monorepos_ grow, Git can struggle to -+manage all that data. As Git commands like `status` and `fetch` get slower, developers stop waiting -+and start switching context. And context switches harm developer productivity. -+ -+`microsoft/git` is focused on addressing these performance woes and making the monorepo developer -+experience first-class. The Scalar CLI packages all of these recommendations into a simple set of -+commands. -+ -+One major feature that Scalar recommends is [partial clone](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/), -+which reduces the amount of data transferred in order to work with a Git repository. While several -+services such as GitHub support partial clone, Azure Repos instead has an older version of this -+functionality called -+[the GVFS protocol](https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md). -+The integration with the GVFS protocol present in `microsoft/git` is not appropriate to include in -+the core Git client because partial clone is the official version of that functionality. -+ -+Downloading and Installing -+========================================================= -+ +If you're working in a monorepo and want to take advantage of the performance boosts in +`microsoft/git`, then you can download the latest version installer for your OS from the +[Releases page](https://github.com/microsoft/git/releases). Alternatively, you can opt to install @@ README.md + +```shell +# Install needed packages -+apt-get install -y curl debsig-verify ++sudo apt-get install -y curl debsig-verify + +# Download public key signature file +curl -s https://github.com/gitapi/repos/microsoft/git/releases/latest \ @@ README.md +gpg --show-keys msft-git-public.asc | head -n 2 | tail -n 1 | tail -c 17 + +# Copy de-armored public key to debsig keyring folder -+mkdir /usr/share/debsig/keyrings/B8F12E25441124E1 -+mv msft-git-public.gpg /usr/share/debsig/keyrings/B8F12E25441124E1/ ++sudo mkdir /usr/share/debsig/keyrings/B8F12E25441124E1 ++sudo mv msft-git-public.gpg /usr/share/debsig/keyrings/B8F12E25441124E1/ + +# Create an appropriate policy file -+mkdir /etc/debsig/policies/B8F12E25441124E1 -+cat > /etc/debsig/policies/B8F12E25441124E1/generic.pol << EOL ++sudo mkdir /etc/debsig/policies/B8F12E25441124E1 ++cat > generic.pol << EOL +<?xml version="1.0"?> +<!DOCTYPE Policy SYSTEM "https://www.debian.org/debsig/1.0/policy.dtd"> +<Policy xmlns="https://www.debian.org/debsig/1.0/"> @@ README.md +</Policy> +EOL + ++sudo mv generic.pol /etc/debsig/policies/B8F12E25441124E1/generic.pol ++ +# Download Debian package +curl -s https://github.com/gitapi/repos/microsoft/git/releases/latest \ +| grep "browser_download_url.*deb" \157: fa45017 = 160: 67da18b scalar: implement a minimal JSON parser
158: 4e1447b = 161: 30bcb31 scalar clone: support GVFS-enabled remote repositories
159: 6106dce = 162: ef7cd18 test-gvfs-protocol: also serve smart protocol
160: e6442c3 = 163: 22ff170 gvfs-helper: add the
endpointcommand161: f80662c = 164: 8d87b2f dir_inside_of(): handle directory separators correctly
162: cee504b = 165: 769c39d scalar: disable authentication in unattended mode
163: ba6dabe = 166: d06dfeb scalar: do initialize
gvfs.sharedCache164: 21af6fb = 167: 05ca72c scalar diagnose: include shared cache info
165: 5e236e5 = 168: 14ce147 scalar: only try GVFS protocol on https:// URLs
166: 3b91ebe = 169: ebece2f scalar: verify that we can use a GVFS-enabled repository
167: 8bfac72 = 170: c068d34 scalar: add the
cache-servercommand168: 6af13a4 = 171: 195b11a scalar: add a test toggle to skip accessing the vsts/info endpoint
169: 5c9a3dd = 172: e6d6640 scalar: adjust documentation to the microsoft/git fork
170: 76037f8 = 173: 6dd1010 scalar: enable untracked cache unconditionally
171: ccb8253 = 174: 4563e4d scalar: parse
clone --no-fetch-commits-and-treesfor backwards compatibility172: 6e88d8c = 175: 7151fbc scalar: make GVFS Protocol a forced choice
173: fae970c = 176: 73ced48 scalar diagnose: accommodate Scalar's Functional Tests
174: 4cc6c2f = 177: fa2be91 ci: run Scalar's Functional Tests
175: ef5bebd = 178: 8dcfc11 scalar: upgrade to newest FSMonitor config setting
178: c8eeb1e = 179: 8c19a36 scalar: configure maintenance during 'reconfigure'
179: 6f76d4a = 180: 0a1116e abspath: make strip_last_path_component() global
180: 94e3ae5 = 181: 27a86ef scalar: .scalarCache should live above enlistment
181: ebb8f0d = 182: 3aa70c7 add/rm: allow adding sparse entries when virtual
182: ab29d91 = 183: 0f85e97 sparse-checkout: add config to disable deleting dirs
183: 827278b = 184: 47d20a2 diff: ignore sparse paths in diffstat
184: 1953200 = 185: 32cd3a0 repo-settings: enable sparse index by default
185: e8418ea = 186: 6ae1b53 diff(sparse-index): verify with partially-sparse
186: a0fe103 = 187: 1ebca13 stash: expand testing for
git stash -u188: bee761d = 188: 259f138 sparse: add vfs-specific precautions
189: e97729c = 189: 1828ff0 reset: fix mixed reset when using virtual filesystem
190: 9c44a89 < -: ------------ Update Ubuntu/Debian steps to use sudo where required
191: 248403c < -: ------------ fixup! release: add Mac OSX installer build
192: 7f7df16 < -: ------------ fixup! release: build unsigned Ubuntu .deb package
193: ed8f523 < -: ------------ fixup! release: create initial Windows installer build workflow
194: 82aa15a < -: ------------ fixup! release: add Mac OSX installer build
195: c68de37 < -: ------------ fixup! release: build unsigned Ubuntu .deb package
196: 5a7c59b < -: ------------ fixup! release: add installer validation
197: 602ec16 < -: ------------ fixup! release: create draft GitHub release with packages & installers
No surprises there, the usual diff context changes as well as applied
fixup!s. The only other thing I did was to turn this into afixup!and applying it:While at it, I also added the respective
Co-authored-by:lines to the commit that was amended, i.e. 3391dff.