Several different operations take too long, sometimes end up crashing. #493

DanielRosenwasser · 2015-10-18T23:41:13Z

git version 2.6.1.windows.1
Windows 10 Enterprise Build 10569 (Insider Preview)

Certain operations are taking absurdly long (running with an SSD, freshly cloned repo). Then, sometimes the operation ends up crashing the program

> git diff
      0 [main] less 5660 C:\Program Files\Git\usr\bin\less.exe: *** fatal error - add_item ("\??\C:\Program Files\Git", "/", ...) failed, errno 1
Stack trace:
Frame        Function    Args
000005FAAE0  0018007208E (00180264C9A, 00180215E59, 00600010000, 000005F8A30)
000005FAAE0  00180046DF2 (000005F9A98, 000005FAAE0, 1D109FC80AE1BA9, 00000000000)
000005FAAE0  00180046E32 (000005F9AB0, 00000000001, 00600010000, 505C3A435C3F3F5C)
000005FAAE0  001800DE09D (00000000000, 000005FCE00, 001800CE718, 1D109FC89A028F7)
000005FCB00  00180127D25 (00000000000, 00300000020, 00000000000, 00300000002)
000005FCBC0  00180047425 (000005FCCA0, 000005FCCA0, 000005FCCA0, 00000000002)
00000000000  0018004602C (00000000000, 000005FCF80, 7FFE76306002, 000C0150008)
00000000000  001800460C4 (00000080088, 00000000000, 00000000000, 00000000000)
00000000000  00100414FC1 (00000000000, 00000000000, 0007E110000, 00000000000)
00000000000  00100401010 (00000000000, 00000000000, 00000000000, 00100401000)
00000000000  7FFE754F2F42 (00000000000, 00000000000, 00000000000, 00000000000)
00000000000  7FFE762D5784 (00000000000, 00000000000, 00000000000, 00000000000)
End of stack trace

Then again

> git push -u origin updateContributingMd
      1 [main] sh 6860 C:\Program Files\Git\usr\bin\sh.exe: *** fatal error - add_item ("\??\C:\Program Files\Git", "/", ...) failed, errno 1
Stack trace:
Frame        Function    Args
000005FAAE0  0018007208E (00180264C9A, 00180215E59, 00600010000, 000005F8A30)
000005FAAE0  00180046DF2 (000005F9A98, 000005FAAE0, 1D109FDB95D207C, 00000000000)
000005FAAE0  00180046E32 (000005F9AB0, 00000000001, 00600010000, 505C3A435C3F3F5C)
000005FAAE0  001800DE09D (00000000000, 000005FCE00, 001800CE718, 1D109FDC2502B8F)
000005FCB00  00180127D25 (00000000000, 00300000020, 00000000000, 00300000002)
000005FCBC0  00180047425 (000005FCCA0, 000005FCCA0, 000005FCCA0, 00000000002)
00000000000  0018004602C (00000000000, 000005FCF80, 7FFE76306002, 000C0150008)
00000000000  001800460C4 (00000080088, 00000000000, 00000000000, 00000000000)
00000000000  001004D24B1 (00000000000, 00000000000, 0007E110000, 00000000000)
00000000000  00100401010 (00000000000, 00000000000, 00000000000, 00100401000)
00000000000  7FFE754F2F42 (00000000000, 00000000000, 00000000000, 00000000000)
00000000000  7FFE762D5784 (00000000000, 00000000000, 00000000000, 00000000000)
End of stack trace

The text was updated successfully, but these errors were encountered:

DanielRosenwasser · 2015-10-19T08:00:05Z

I actually cannot open Git Bash at all.

dscho · 2015-10-19T08:32:34Z

I am using Git for Windows without problems on Windows 10 Enterprise (10.0.10240). Comparing my experience with your account leads me to conclude that your setup has at least one quirk that mine lacks.

This conclusion in turn leads me to the observation that this bug report hides all details about the setup except the Windows version. It does not even mention whether this is a 32-bit or a 64-bit Windows. Please do not do that. It makes it hard for others to help you, even if they want. To help you come up with the necessary information, I encourage you to follow the advice given in https://github.com/git-for-windows/git/wiki/Issue-reporting-guidelines to provide enough detail.

dscho · 2015-10-19T19:08:02Z

Well, the closest I can do to help without more details as to the setup is to refer you to http://stackoverflow.com/questions/28861237/bash-is-crashing-on-cygwin-add-item-c-cygwin and wish you the best of luck 😺

DanielRosenwasser · 2015-10-19T19:47:21Z

Hey @dscho sorry about that. I didn't mean to dump an error and leave you with that. Totally not my intention. I wasn't familiar with the error reporting guidelines, so I'm glad you pointed them out.

I'm running on a 64-bit version of Windows if that helps at all.

Something I found on the bus today is that when not connected to a network, I don't observe the same issues at all. When I'm connected directly into my office network, I don't observe the issue either. This was all seen on my home network.

Could it be an issue related to being on a domain-joined computer? My machine uses my office's domain, and I remember having certain issues with Cygwin related to that.

dscho · 2015-10-19T21:16:02Z

Something I found on the bus today is that when not connected to a network, I don't observe the same issues at all. When I'm connected directly into my office network, I don't observe the issue either. This was all seen on my home network.

Interesting... So is your domain controller simply not reachable in that case?

DanielRosenwasser · 2015-10-19T23:37:34Z

That appears to be the issue, yes. Git should ideally be resilient to this though, right?

DanielRosenwasser · 2015-10-20T08:06:31Z

This is indeed amended by the suggested fix on the Cygwin website. For anyone searching for a solution, I've summed up the steps below in case anyone else is running into excruciatingly slow startups/crashes in Cygwin and/or Git for Windows.

Still, I feel like the tools shouldn't crash in this situation. At the very least, users need to have the getent binary in case they run into something like this.

To clarify how I got around this

Install Cygwin

From a Cygwin Bash prompt, type in the following

getent passwd $(id -u) > /etc/passwd
getent group $(id -G) > /etc/group

Assuming that your Cygwin installation was C:\cygwin\ and that your Git for Windows installation is at C:\Program Files\Git\:
1. Open Windows Explorer or an elevated PowerShell window.
2. Copy C:\cygwin\etc\passwd to C:\Program Files\Git\etc\passwd
3. Copy C:\cygwin\etc\group to C:\Program Files\Git\etc\group

dscho · 2015-10-20T09:01:03Z

Please note that this solution works correctly only if you are working on a single-user machine.

What does your /etc/nsswitch.conf show? It should contain these lines:

# Begin /etc/nsswitch.conf

passwd: files db
group: files # db

db_enum: cache builtin

db_home: env windows cygwin desc
db_shell: env windows # cygwin desc
db_gecos: env # cygwin desc

# End /etc/nsswitch.conf

I worked quite long days just to fix this vexing issue. It is a bummer that it crops up again. But still, we should fix it so it also works on multi-user machines.

kostix · 2015-10-20T14:45:45Z

@DanielRosenwasser, see #193

dscho · 2015-11-09T19:07:36Z

What does your /etc/nsswitch.conf show?

@DanielRosenwasser I am still waiting for an answer.

DanielRosenwasser · 2015-11-12T09:06:51Z

Sorry about that - currently it shows

# Begin /etc/nsswitch.conf

passwd: files db
group: files # db

db_enum: cache builtin

db_home: env windows cygwin desc
db_shell: env windows # cygwin desc
db_gecos: env # windows cygwin desc

# End /etc/nsswitch.conf

DanielRosenwasser · 2015-11-12T09:15:28Z

So Cygwin itself was running into this problem again on my computer. Running the program which took an absurd amount of time. I was able to remedy it by changing Cygwin's nsswitch.conf from

# /etc/nsswitch.conf
#
#    This file is read once by the first process in a Cygwin process tree.
#    To pick up changes, restart all Cygwin processes.  For a description
#    see https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-mapping-nsswitch
#
# Defaults:
# passwd:   files db
# group:    files db
# db_enum:  cache builtin
# db_home:  /home/%U
# db_shell: /bin/bash
# db_gecos: <empty>

# /etc/nsswitch.conf
#
#    This file is read once by the first process in a Cygwin process tree.
#    To pick up changes, restart all Cygwin processes.  For a description
#    see https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-mapping-nsswitch
#
# Defaults:
# passwd:   files db
 group:    files # db
# db_enum:  cache builtin
# db_home:  /home/%U
# db_shell: /bin/bash
# db_gecos: <empty>

Sorry if this is useless information to you. Let me know if there's anything I can further diagnose to help you.

dscho · 2015-11-12T09:48:22Z

So you set the group in Cygwin as it is in Git for Windows and that fixed it in Cygwin, but not in Git for Windows? Sorry, things are still unclear to me.

DanielRosenwasser · 2015-11-13T08:23:45Z

I already fixed the issue in Git for Windows by using the getent command and creating a group and passwd file. If I remove the passwd file, then I'll get the same issues.

I started seeing the same slowdown symptoms in Cygwin, so I

Used the same group and passwd files from Git for Windows
Changed the group option in my nsswitch.conf from group files db to group: files.

dscho · 2015-11-13T18:10:23Z

I already fixed the issue in Git for Windows by using the getent command and creating a group and passwd file.

Unfortunately, as I pointed out, this is not a fix but a work-around: it only happens to work in a single-user installation.

I started seeing the same slowdown symptoms in Cygwin

My question stands: with your edits (that imitate Git for Windows' /etc/nsswitch.conf), is Cygwin fast again?

DanielRosenwasser · 2015-11-13T18:12:43Z

Yes, with my changes Cygwin is fast again.

dscho · 2015-11-13T18:24:44Z

@DanielRosenwasser I hate to put too much load on other's shoulders, but at the moment, I have no choice because I am awfully short on time.

I would like to suggest to rebuild the MSys2 runtime, after calling

cd /usr/src/MSYS2-packages/msys2-runtime/src/msys2-runtime
git pull https://github.com/Alexpux/Cygwin msys2-master

(this requires the msys2-runtime to be built at least once before).

The idea is to test whether there has been a fix in upstream Cygwin that we do not have in Git for Windows' fork yet.

dscho · 2016-02-15T16:11:59Z

@DanielRosenwasser are you still interested in this issue?

vitalyster · 2016-02-17T12:45:44Z

Is it possible to throw away this Cygwin bloat and ask Windows about users and groups directly? There was no such problem in gfw 1.9.x

dscho · 2016-02-17T14:05:17Z

Is it possible to throw away this Cygwin bloat [...]

Sure. Are you going to do that? ;-)

vitalyster · 2016-02-17T14:45:15Z

Yes, I can help. I still can't understand why git for windows declared as native windows app (and it was before 2.x) now rely on stripped-down poor-maintained cygwin fork. Why not just use cygwin?

dscho · 2016-02-17T15:37:39Z

I still can't understand why git for windows declared as native windows app (and it was before 2.x) now rely on stripped-down poor-maintained cygwin fork.

See https://github.com/git-for-windows/git/wiki#about for some background.

Why not just use cygwin?

Speed.

dscho · 2016-04-02T12:27:27Z

Okay, this ticket limped along enough.

…sing virtual filesystem This replaces git-for-windows#493 (can't reopen a PR after a force-push...). I updated this commit with a more firm version of the fix. This hopefully answers Victoria's excellent concerns with the previous approach. I did not manage to get an automated test for this, but I did carefully verify this manually with a few commits in a VFS for Git enlistment (with different files every time). I updated the commit message with more details about why this works. --- This fork contains changes specific to monorepo scenarios. If you are an external contributor, then please detail your reason for submitting to this fork: * [X] This change only applies to the virtualization hook and VFS for Git. Resolves git-for-windows#490.

dscho added pending-answer unclear labels Oct 19, 2015

dscho mentioned this issue Feb 15, 2016

Fatal error in sh.exe when attempting to checkout submodules from TeamCity #659

Closed

dscho closed this as completed Apr 2, 2016

bcoov mentioned this issue Apr 12, 2017

fatal error - add_item(...) with ssh.exe from Git for Windows 2.8.1+ #1135

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Several different operations take too long, sometimes end up crashing. #493

Several different operations take too long, sometimes end up crashing. #493

DanielRosenwasser commented Oct 18, 2015

DanielRosenwasser commented Oct 19, 2015

dscho commented Oct 19, 2015

dscho commented Oct 19, 2015

DanielRosenwasser commented Oct 19, 2015

dscho commented Oct 19, 2015

DanielRosenwasser commented Oct 19, 2015

DanielRosenwasser commented Oct 20, 2015

dscho commented Oct 20, 2015

kostix commented Oct 20, 2015

dscho commented Nov 9, 2015

DanielRosenwasser commented Nov 12, 2015

DanielRosenwasser commented Nov 12, 2015

dscho commented Nov 12, 2015

DanielRosenwasser commented Nov 13, 2015

dscho commented Nov 13, 2015

DanielRosenwasser commented Nov 13, 2015

dscho commented Nov 13, 2015

dscho commented Feb 15, 2016

vitalyster commented Feb 17, 2016

dscho commented Feb 17, 2016

vitalyster commented Feb 17, 2016

dscho commented Feb 17, 2016

dscho commented Apr 2, 2016

Several different operations take too long, sometimes end up crashing. #493

Several different operations take too long, sometimes end up crashing. #493

Comments

DanielRosenwasser commented Oct 18, 2015

DanielRosenwasser commented Oct 19, 2015

dscho commented Oct 19, 2015

dscho commented Oct 19, 2015

DanielRosenwasser commented Oct 19, 2015

dscho commented Oct 19, 2015

DanielRosenwasser commented Oct 19, 2015

DanielRosenwasser commented Oct 20, 2015

To clarify how I got around this

dscho commented Oct 20, 2015

kostix commented Oct 20, 2015

dscho commented Nov 9, 2015

DanielRosenwasser commented Nov 12, 2015

DanielRosenwasser commented Nov 12, 2015

dscho commented Nov 12, 2015

DanielRosenwasser commented Nov 13, 2015

dscho commented Nov 13, 2015

DanielRosenwasser commented Nov 13, 2015

dscho commented Nov 13, 2015

dscho commented Feb 15, 2016

vitalyster commented Feb 17, 2016

dscho commented Feb 17, 2016

vitalyster commented Feb 17, 2016

dscho commented Feb 17, 2016

dscho commented Apr 2, 2016