git status always show random files are modified on Win7-x64/SSD driver.

[lygstate]

I reset the whole source tree everytime, and even delete the whole folder, and clone the new one
It's always should some file are modified, but the fact, it doesn't,
That's very annoy,

  （使用 "git add <file>..." 更新要提交的内容）
  （使用 "git checkout -- <file>..." 丢弃工作区的改动）

        修改：     xpcom/sample/nsSample.manifest
        修改：     xpcom/tests/unit/bug725015.manifest
        修改：     xpcom/tests/unit/data/SmallApp.app/Contents/Resources/English.lproj/MainMenu.nib/keyedobjects.nib
        修改：     xpcom/tests/unit/data/bug121341.properties
        修改：     xpcom/tests/unit/data/iniparser02.ini
        修改：     xpcom/tests/unit/data/iniparser03.ini
        修改：     xpcom/tests/unit/data/iniparser04.ini
        修改：     xpcom/tests/unit/data/iniparser05.ini
        修改：     xpcom/tests/unit/data/iniparser06.ini
        修改：     xpcom/tests/unit/data/iniparser07.ini
        修改：     xpcom/tests/unit/data/iniparser08.ini
        修改：     xpcom/tests/unit/data/iniparser09.ini
        修改：     xpcom/tests/unit/data/iniparser10.ini
        修改：     xpcom/tests/unit/data/iniparser11.ini
        修改：     xpcom/tests/unit/data/iniparser12.ini
        修改：     xpcom/tests/unit/data/iniparser13.ini
        修改：     xpcom/tests/unit/data/iniparser14.ini
        修改：     xpcom/tests/unit/data/iniparser15.ini
        修改：     xpcom/tests/unit/data/presentation.key/thumbs/st0.tiff
        修改：     xpcom/tests/unit/test_bug725015.js
        修改：     xpcom/threads/HangMonitor.h
        修改：     xulrunner/app/install_app.py
        修改：     xulrunner/installer/windows/Header.bmp

[t-b]

Well I can't reproduce that here.
Does it make a difference if you have the repository on the ssd or not?

[DarkNami]

Maybe it is 'line endings' issue ?

Reference:

• Dealing with line endings
  https://help.github.com/articles/dealing-with-line-endings/
• Git 在 Windows 平台處理斷行字元 (CRLF) 的注意事項
  http://blog.miniasp.com/post/2013/09/16/Git-for-Windows-Line-Ending-Conversion-Notes.aspx

[haimgel]

I have a very similar problem: after "git clone", different set of files is reported as "modified". This is related to CRLF conversion for sure -- all files in question are defined in .gitattributes, and some of them may have been comitted with CRLF endings.

The real problem, as I see it, is that the results are inconsistent. After, say, 10 runs of "git clone" and "git status", about 25% reports are "clean", and the rest report some files as changed. But the number of files changes from run to run. Really confusing.

This is not related to SSDs, at least for me. Happens on SSDs and on normal hard disks. I do have Windows 7 x64, though.

[linquize]

Does it happen in v1.9.4.msysgit.2?

[haimgel]

Yes, I'm using the latest version.

[YueLinHo]

@hageltech
What happen if using git clone with --no-checkout (-n) option and then perform the checkout after the clone finished?

[haimgel]

I tried with -n and then did checkout after the clone finished. Still the same problem. Some checkouts were dirty, some were not. Moreover, I did one more test: did one "checkout -n", then copied the bare repository several times using simple filesystem copy, and did checkout in each of them. Some were dirty, some were not, again. So the problem seems to be in the checkout, not in clone.

[YueLinHo]

Perhaps, try to locate one modified file and use git check-attr -a <that_file_pathname>

[haimgel]

Did in both trees (where the file is ok and where it is reported as modified). Both return the same:
EmailTestDlg.dfm: text: set
EmailTestDlg.dfm: eol: crlf

This is what I have in .gitattributes for this extension:
*.dfm text eol=crlf

This is what I have in gitconfig (in core section, related to this issue, possibly):
autocrlf = true
eol = crlf

[YueLinHo]

Then, what is the EOL of that file actually? LF?

[haimgel]

Nope, that's the weird thing: both cases, file is correctly CRLF in the working directory. However, it is shown as "modified" sometimes, and when it does, I cannot make that status go away by any means I know (.e.g git reset --hard, etc.)

[YueLinHo]

Could you please try to create a dummy problematic repository?

[kusma]

@hageltech: Do you have a virus scanner running in the background?

[haimgel]

@kusma: I have Microsoft Security Essentials running... I have hard time believing it is to blame, though.
@YueLinHo: I'll try doing that later today.

[haimgel]

@YueLinHo: OK, I couldn't cause this issue to appear with very small dummy testing repositories. However, I did this:

I took one existing repository I have no problem sharing (it's open source), added my .gitattributes file there, and the issue started to appear. So:

Clone this repo:
git clone https://github.com/hageltech/msgit_test.git

Then clone the repo locally several times (at least 10 times to be sure):
git clone msgit_test 1
git clone msgit_test 2
git clone msgit_test 3
...

See how "git status | wc -l" reports different number of files as changed in "1", "2", ...

I just run these tests in a clean Windows 7 in a VM, and the issue is there, so it's not something else on my machine that is interfering.

[YueLinHo]

@hageltech

I tested it, it happened at 4th clone.
Then, I review the last commit 371f51f of testing repo.
Seems forget to do the normalization, so I re-do it by these steps:

1. Add .gitattributes
2. Delete all files execpte the .gitattributes
3. Revert all deleted files
4. commit .gitattributes and all normalized files

See https://github.com/YueLinHo/msgit_test
Then, re-test it for 20 times. no problem anymore.

As @DarkNami mention:
see Dealing with line endings
Especially the Refreshing a repository after changing line endings part.

[lygstate]

Finally have this issue to be watched, I have a small group of people using msysgit 1.9.4 that always facing this problem, that's really annoying in daily life. There is nothing problem with the MSE.

[haimgel]

@YueLinHo: Thank you very much for pointing me to the workaround. My main concern is:

• msgit behaves unpredictably. When "git clone" give different results from run to run, it's not ok, normalization or not normalization. This is the reason it gets confusing, because everything appears to work ok for a while, and then after say 5 commits and clones you get "modified files" problem. it's not trivial to attribute the issue to .gitattributes commit, especially when everything worked ok for a while.
• msgit behaves differently from the original git. Git always lists all affected files as "modified", and msgit produces such result only occasionally.

So, in my opinion, it's still a regression.

[YueLinHo]

@hageltech

msgit behaves unpredictably. When "git clone" give different results from run to run, it's not ok, normalization or not normalization

I suppose that it will be not happened if you commit .gitattributes with the normalization.
Otherwise, you will get into that situation as you said, but you still can do the normalization after 5 commits is commit into repository. Then, I also suppose it will be not happened again.

In the other hand, indeed, it is a issue that committing .gitattributes without the normalization leads to that git does not list all files which should be normalized for each new clone.

I guess the root cause is on the timing of refresh index.

BTW, the normalization steps I tested and listed in this comment are via TortoiseGit.
(Also try CLI once and it works too.)

[haimgel]

@YueLinHo: Additional info:

Normalization does not work as intended

Steps to reproduce:

Clone my test repo:
git clone https://github.com/hageltech/msgit_test.git

Try to normalize as you suggested above:
git rm --cached -r . & git reset --hard & git status

Each time you run it, you get a different result. Sometimes files are listed as changed, sometimes they are not. End result is, this is not reliable with msgit.

I suggest you remove the "worksforme" tag... This seems to be a reproducible, serious issue with msgit. No way to normalize line ends with msgit, I had to use git on Linux to be sure all files are processed.

[YueLinHo]

Have you deleted all files first?

[haimgel]

Yep, I just changed the command to:
rm -Rf * & git rm --cached -r . & git reset --hard & git status

Same result. Sometimes files are listed as changed, sometimes "working directory clean".

Just run the above over and over...

[YueLinHo]

make sure only the .gitattributes is not deleted?

[haimgel]

Yes, @YueLinHo, I verified that manually also. Only .git directory and .gitattributes remain after "rm -Rf *".

[YueLinHo]

:-/
I test 14 times, 6 of them do not list all modified file. but all(6) of them show 63 modified files after normalization.
(The rest 8 shows 63 modified files directly.)

(using 1.9.2.0)

[haimgel]

I don't quite follow what you're saying... I'm implying that the normalization process itself (e.g. delete all files, do git reset") is flaky and does not always normalize all files. That the clone was doing weird things is just a symptom of this root problem.

Can you reproduce this or not? Do you agree normalization does not work as intended?

[YueLinHo]

Upgrade to 1.9.4.2 and re-test it

1. cloned your testing repo to 20 repo
2. only 1 local repo, named it A repo, shows 51 modified files, the others show 63 modified files. (expect 63 modified files)
3. do normalize in A. Then, it shows 63 modified files as I expect.

I can reproduce the problem that it shows different number of modified for each new clone.
I can not reproduce the situation that normalization is failed.
So, based on my tests, I must to say that the normalization works for me.

I can/will do more test. ^_^

(BTW, I ever do a dangerous test that delete .git\index directly, then perform reset mixed.
Still works. Did this test, because I don't like to delete all file in working tree. not make sense for large repo.)

[YueLinHo]

@hageltech
SRROY. I agree normalization does not work always.
At least, the following steps does not work always:

1. delete all files (except .gitattributes and .git folder)
2. "git rm --cached -r ."
3. "git reset --hard"

The following is another 75 testing result:
I cloned another 75 repo.
49 of 75: shows as I expect
26 of 75: do not.
then
3 of 26: normalization does not works at first time. (but second try.)

I feeling it might be some racing condition.
does the normalization always work in Linux?

(BTW, I don't have the right to remove "WorksForMe" tag. me, a user too.)

[haimgel]

I wrote a little script to do "rm -Rf * && git rm --cached -r . && git reset --hard & git status" repeatedly on Linux, with git 2.0.0, and the results are inconsistent too!

It behaves a little different, though... After around 30-40 "good" iterations I get one bad one, where working directory is listed as "clean".

So the problem is much less pronounced on Linux, but I must say, at this point, that the issue seems to be with Git in general, and not msysgit specific. Although this is more visible on Windows, the proper course of action is to check whether this still occurs with the latest version of git, and if yes, then to raise this issue at git mailing list.

[YueLinHo]

agree that

[lygstate]

@t-b , Hello, can you remove WorksForMe for this issue.

[haimgel]

I took this issue to Git mailing list, since it is not restricted to msysgit (although it is more pronounced on msysgit):

http://thread.gmane.org/gmane.comp.version-control.git/259202

[dscho]

Thanks. Closing here.

[YueLinHo]

Could you please also change the label from WorksForMe to Upstream? ^_^ Thank you.

[lygstate]

Which is the upstream bug? This is not enough.

[YueLinHo]

The conclusion
@DarkNami provides some references, such as Dealing with line endings, and might solve the problem.
@hageltech provides more information and a dummy repo which try to figure out what happened, also did a lot of tests for Windows and Linux.

These lead to:
Based on Dealing with line endings,
the normalization does not work ALWAYS(but usually), even on Linux OS.

So, @hageltech reports it to upstream.
I agree that, and I suppose @dscho also agree that.

---

@lygstate
If these are not enough, I suppose you need to do something enough to reopen this issue.
(Have you tried the normalization? Perhaps you experience different problem.)
Thank you. :)

[linquize]

But random is strange.

[YueLinHo]

@linquize
yeah, that is the problem.

[lygstate]

Please don't close this issue by now, the git community doesn't response yet.

[dscho]

@lygstate do I understand correctly that you are offering to put in the effort it takes to debug the problem properly?

[lygstate]

@dscho I means the git community doesn't response for this issue, so should we re-ping for this issue?

[dscho]

@lygstate I think that the only way you can get what you want is to actually invest your own time, effort and/or money into it. If you expect others to do your bidding, you might get lucky, but there is no guarantee. This is my personal opinion, of course.

[lygstate]

@dscho You are totally talking about another thing, I means this issue should be re-open to tracking it.

[YueLinHo]

@lygstate
It's the same thing. It means you need to do something, bring up enough information to re-open this issue.

[lygstate]

I've done the conversation, bye