Bug 575393 - Reset Hard does not reliably perform the reset
Summary: Reset Hard does not reliably perform the reset
Status: RESOLVED FIXED
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: 5.12   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 6.0   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-08-13 06:03 EDT by Michael Haubenwallner CLA
Modified: 2021-10-30 17:32 EDT (History)
3 users (show)

See Also:


Attachments
Screenshot (193.63 KB, image/png)
2021-10-12 02:06 EDT, Jörg Kubitz CLA
no flags Details
screenshot git diff (37.35 KB, image/png)
2021-10-12 02:48 EDT, Jörg Kubitz CLA
no flags Details
file1.java (79.74 KB, application/octet-stream)
2021-10-12 02:50 EDT, Jörg Kubitz CLA
no flags Details
File2.java (77.53 KB, application/octet-stream)
2021-10-12 02:50 EDT, Jörg Kubitz CLA
no flags Details
screenshot NewWizardMessages (76.83 KB, image/png)
2021-10-29 02:41 EDT, Jörg Kubitz CLA
no flags Details
NewWizardMessages - On Disk.properties (74.94 KB, application/octet-stream)
2021-10-29 08:51 EDT, Jörg Kubitz CLA
no flags Details
NewWizardMessages - after git reset --hard.properties (73.75 KB, application/octet-stream)
2021-10-29 08:52 EDT, Jörg Kubitz CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Haubenwallner CLA 2021-08-13 06:03:31 EDT
So I got a situation where Git Reset Hard (via Eclipse) does not help, but commandline 'git reset --hard' does.

What I did:
* Some local code changes, with additional source files
* Close each source code editor
In Git Staging View:
* Sort by state
* Select all modified files, Replace with HEAD revision, auto build starts building
* Select all untracked files, Delete, while auto build is still building

What I have now:
* Compilation errors because some of my local changes to already tracked source files are still there (checked with external editor)
* Git Staging View does not show any local changes, neither to workspace nor index
* Commandline 'git status' and 'git diff' both do _not_ show any changes
* Commandline 'git blame' on that files _does_ show 'Not Committed Yet' lines

What I tried:
* Eclipse Git Reset Hard: does not change anything
* Commandline 'git reset --hard': does reset the source files to HEAD

This is with EGit and JGit both of version 5.12.0.202106070339-r.

Thanks!
Comment 1 Jörg Kubitz CLA 2021-10-12 02:01:54 EDT
I currently have a similar problem where the staging view shows one file as modified. But "Replace with HEAD Revision" does not change that.
@Thomas if it helps you could take a look at my computer or get any sort of dump that could help solving the issue today.
Comment 2 Jörg Kubitz CLA 2021-10-12 02:06:04 EDT
Created attachment 287297 [details]
Screenshot
Comment 3 Thomas Wolf CLA 2021-10-12 02:27:19 EDT
Check for mixed line endings, also in the index. Or check for the executable bit -- is it set in the index?

Sorry, I won't have time for a screen sharing session.
Comment 4 Jörg Kubitz CLA 2021-10-12 02:48:22 EDT
Created attachment 287298 [details]
screenshot git diff

"git diff" from "git bash" shows something that i would interprete as different line endings on many but 1 line.
I am using windows and autocrlf is set to true.
I do not know where i can see the "executable bit".
My expectation is that 1) there is a option to see the difference in eclipse. 2) "replace with head" will fix it.

$ git diff  Test_org_eclipse_swt_browser_Browser.java
diff --git a/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_eclipse_swt_                                                                                                                                                                         browser_Browser.java b/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_ec                                                                                                                                                                         lipse_swt_browser_Browser.java
index e5bd6663d2..2a80821961 100644
--- a/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_eclipse_swt_browser                                                                                                                                                                         _Browser.java
+++ b/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_eclipse_swt_browser                                                                                                                                                                         _Browser.java
@@ -1,2269 +1,2269 @@
-/*******************************************************************************
- * Copyright (c) 2000, 2018 IBM Corporation and others.
- *
- * This program and the accompanying materials
- * are made available under the terms of the Eclipse Public License 2.0
- * which accompanies this distribution, and is available at
- * https://www.eclipse.org/legal/epl-2.0/
- *
- * SPDX-License-Identifier: EPL-2.0
- *
- * Contributors:
- *     IBM Corporation - initial API and implementation
- *******************************************************************************/
-package org.eclipse.swt.tests.junit;
-
-import static org.eclipse.swt.browser.LocationListener.changedAdapter;
-import static org.eclipse.swt.browser.LocationListener.changingAdapter;
:...skipping...
diff --git a/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_eclipse_swt_browser_Browser.java b/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_eclipse_swt_browser_Browser.java
index e5bd6663d2..2a80821961 100644
--- a/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_eclipse_swt_browser_Browser.java
+++ b/tests/org.eclipse.swt.tests/JUnit Tests/org/eclipse/swt/tests/junit/Test_org_eclipse_swt_browser_Browser.java
@@ -1,2269 +1,2269 @@



"git checkout -- Test_org_eclipse_swt_browser_Browser.java" from "git bash" did resolve the issue in eclipse.
Comment 5 Jörg Kubitz CLA 2021-10-12 02:50:02 EDT
Created attachment 287299 [details]
file1.java
Comment 6 Jörg Kubitz CLA 2021-10-12 02:50:57 EDT
Created attachment 287300 [details]
File2.java

even the file size did differ
Comment 7 Jörg Kubitz CLA 2021-10-12 02:55:14 EDT
Fun fact: Notepad++'s "Compare" plugin also states the files have no differences even though one is marked as DOS\Windows and the other as UNIX line endings.
Comment 8 Thomas Wolf CLA 2021-10-12 09:34:29 EDT
Jörg, which file is which?

File2 is generally LF terminated, except at line 89

  boolean isChromium = false, isEdge = false;

which is terminated by CR-LF. (The byte at offset 0xE1F is the CR.)

Now it really matters which file was the one checked out with JGit, and which was checked out with git.
Comment 9 Jörg Kubitz CLA 2021-10-12 09:42:59 EDT
(In reply to Thomas Wolf from comment #8)

> Jörg, which file is which?
File1 (filename includes "kopie") was the file on my drive before i used commandline reset. I do not know if it was checked out in that form by jgit. It may be that eclipse did touch it with its editor. But i doubt that because i could just reproduce that error with the same file on another copuet by simply pulling  https://git.eclipse.org/r/platform/eclipse.platform.swt

File2 was the file in the repository that was created by git checkout --.
Comment 10 Jörg Kubitz CLA 2021-10-12 09:45:07 EDT
(In reply to Jörg Kubitz from comment #9)
> (In reply to Thomas Wolf from comment #8)
> on another copuet 

"on another computer" i meant

Thomas, thanks for looking on it.
Comment 11 Thomas Wolf CLA 2021-10-12 09:57:42 EDT
So it looks like that file has been committed with that CR at some point, and JGit has a code path somewhere that somehow doesn't find that CR in the index blob.

git does not apply any LF-CRLF transformations if it finds that the blob in the repository contains a CR. JGit should do the same (I thought I had fixed that a while ago), but apparently we're missing a case somewhere.
Comment 12 Eclipse Genie CLA 2021-10-12 16:18:25 EDT
New Gerrit change created: https://git.eclipse.org/r/c/jgit/jgit/+/186407
Comment 13 Eclipse Genie CLA 2021-10-12 16:42:39 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.swt/+/186410
Comment 14 Thomas Wolf CLA 2021-10-12 16:55:34 EDT
The JGit commit fixes this; the SWT commit cleans up the SWT repository to get rid of mixed line endings.
Comment 17 Jörg Kubitz CLA 2021-10-29 02:41:02 EDT
Created attachment 287393 [details]
screenshot NewWizardMessages

Today i have the same problem with

org.eclipse.jdt.ui/ui/org/eclipse/jdt/internal/ui/wizards/NewWizardMessages.properties

I wonder why i still can not "Replace with HEAD revision" - I have Java implementation of Git 6.0.0.202110270955-m2 org.eclipse.jgit.feature.group running.

"git diff" from command line shows differences (see screenshot) but "Replace with HEAD revision" in eclipse does nothing.
Comment 18 Thomas Wolf CLA 2021-10-29 07:49:46 EDT
(In reply to Jörg Kubitz from comment #17)
> Created attachment 287393 [details]
> screenshot NewWizardMessages
> 
> Today i have the same problem with
> 
> org.eclipse.jdt.ui/ui/org/eclipse/jdt/internal/ui/wizards/NewWizardMessages.
> properties
> 
> I wonder why i still can not "Replace with HEAD revision" - I have Java
> implementation of Git 6.0.0.202110270955-m2 org.eclipse.jgit.feature.group
> running.
> 
> "git diff" from command line shows differences (see screenshot) but "Replace
> with HEAD revision" in eclipse does nothing.

What is this screenshot showing? What is being compared here? What are all the relevant CRLF settings? (core.autocrlf and core.eol, but also any definitions for text or eol in .gitattributes files, if any). What is the repository state (resolving conflicts? or normal?) What's the HEAD commit?

With just this screenshot I cannot even begin to analyze what the problem might be.
Comment 19 Jörg Kubitz CLA 2021-10-29 08:51:09 EDT
(In reply to Thomas Wolf from comment #18)
> What is this screenshot showing? 

The screenshot "git bash" console in the foreground. There i executed "git diff NewWizardMessages.properties". One can see line ending changed and in one line there is also a big red square. Probably a whitespace was changed there.
I did not touch the file manually. I only pulled.
In the background / in the bottom there is an eclipse window which shows that the file is in the Git staging view.

> What is being compared here? 

I guess git diff shows the difference between NewWizardMessages.properties on the disk vs in the index.

> What are all the relevant CRLF settings? (core.autocrlf and core.eol, but also any
> definitions for text or eol in .gitattributes files, if any).
[core]
	symlinks = false
	repositoryformatversion = 0
	filemode = false
	logallrefupdates = true
	autocrlf = true



> What is the repository state (resolving conflicts? or normal?)

normal.

> What's the HEAD commit?
commit eeb1f37fc0731f52c310b58e492bb20686549e9a
Comment 20 Jörg Kubitz CLA 2021-10-29 08:51:42 EDT
Created attachment 287396 [details]
NewWizardMessages - On Disk.properties
Comment 21 Jörg Kubitz CLA 2021-10-29 08:52:14 EDT
Created attachment 287397 [details]
NewWizardMessages - after git reset --hard.properties
Comment 22 Jörg Kubitz CLA 2021-10-29 08:55:52 EDT
after git reset --hard from command line the file was gone from staging view.
Then touched it, saved it. -> it shwos up in the staging view again.
But again  "Replace with HEAD revision" does not remove it from the view. it stay dirty again.
Comment 23 Thomas Wolf CLA 2021-10-29 19:17:12 EDT
(In reply to Jörg Kubitz from comment #22)
> after git reset --hard from command line the file was gone from staging view.
> Then touched it, saved it. -> it shwos up in the staging view again.
> But again  "Replace with HEAD revision" does not remove it from the view. it
> stay dirty again.

I'll take a look. The red thing in your screenshot is two trailing blanks on that line. No idea where those come from.
Comment 24 Thomas Wolf CLA 2021-10-29 19:35:10 EDT
(In reply to Jörg Kubitz from comment #21)
> Created attachment 287397 [details]
> NewWizardMessages - after git reset --hard.properties

So this is a simple problem. The file is generally LF terminated, but contains 6 CR-LFs, the first at offset 0x261F.

I was waiting for this. (Really I was. I *knew* it would happen some day.)

JGit, for whatever reasons, checks only the first 8kB of a file for CRs. Thats 0x2000. The first 8kB of this file are all properly LF terminated. JGit never sees these CRs.

git, OTOH, appears to check the whole file. (And also its logic has evolved a little bit, whereas JGit appears to use simpler older logic. For instance, git considers files containing *lone* CRs as binary, in addition to files containing NUL (0x00) bytes. JGit only considers NUL.)

The quick fix here is to increase that size, or also scan the whole blob. The next fix would then be to ensure JGit uses the same logic as C git.
Comment 25 Thomas Wolf CLA 2021-10-30 09:34:23 EDT
I've factored this out into bug 576971. Let's continue there, instead of this already closed bug.
Comment 26 Eclipse Genie CLA 2021-10-30 17:32:37 EDT
New Gerrit change created: https://git.eclipse.org/r/c/egit/egit/+/187195