Bug 579116 - Error occasionally when pulling multiple repositories
Summary: Error occasionally when pulling multiple repositories
Status: RESOLVED FIXED
Alias: None
Product: EGit
Classification: Technology
Component: UI (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 10
: P3 normal (vote)
Target Milestone: 6.2   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-07 00:54 EST by Kenneth Styrberg CLA
Modified: 2022-03-20 05:56 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kenneth Styrberg CLA 2022-03-07 00:54:10 EST
Eclipse 2022-03 RC 1, Windows 10

I occasionally get this error. Eclipse is setup to run 3 parallel pulls

eclipse.buildId=4.23.0.I20220223-1800
java.version=17.0.2
java.vendor=Eclipse Adoptium
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=en_US
Framework arguments:  -refresh -product org.eclipse.epp.package.rcp.product
Command-line arguments:  -os win32 -ws win32 -arch x86_64 -clean -refresh -product org.eclipse.epp.package.rcp.product

org.eclipse.egit.ui
Error
Mon Mar 07 06:44:11 CET 2022
C:\development\git\XXX\.git\HEAD (The process cannot access the file because it is being used by another process)

java.io.FileNotFoundException: C:\development\git\XXX\.git\HEAD (The process cannot access the file because it is being used by another process)
	at java.base/java.io.FileInputStream.open0(Native Method)
	at java.base/java.io.FileInputStream.open(FileInputStream.java:216)
	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
	at org.eclipse.jgit.util.io.SilentFileInputStream.<init>(SilentFileInputStream.java:31)
	at org.eclipse.jgit.util.IO.readSome(IO.java:68)
	at org.eclipse.jgit.internal.storage.file.RefDirectory.scanRef(RefDirectory.java:1096)
	at org.eclipse.jgit.internal.storage.file.RefDirectory.readRef(RefDirectory.java:1050)
	at org.eclipse.jgit.internal.storage.file.RefDirectory.readAndResolve(RefDirectory.java:264)
	at org.eclipse.jgit.internal.storage.file.RefDirectory.exactRef(RefDirectory.java:286)
	at org.eclipse.jgit.lib.Repository.exactRef(Repository.java:1078)
	at org.eclipse.egit.ui.internal.RepositoryStateCache.lambda$2(RepositoryStateCache.java:99)
	at org.eclipse.egit.core.UnitOfWork.get(UnitOfWork.java:132)
	at org.eclipse.egit.ui.internal.RepositoryStateCache.getHead(RepositoryStateCache.java:94)
	at org.eclipse.egit.ui.internal.RepositoryStateCache.lambda$6(RepositoryStateCache.java:168)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	at org.eclipse.egit.ui.internal.RepositoryStateCache.getHeadRef(RepositoryStateCache.java:167)
	at org.eclipse.egit.ui.internal.decorators.DecoratorRepositoryStateCache.lambda$1(DecoratorRepositoryStateCache.java:112)
	at org.eclipse.egit.core.UnitOfWork.get(UnitOfWork.java:132)
	at org.eclipse.egit.ui.internal.decorators.DecoratorRepositoryStateCache.lambda$0(DecoratorRepositoryStateCache.java:111)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	at org.eclipse.egit.ui.internal.decorators.DecoratorRepositoryStateCache.getCurrentBranchLabel(DecoratorRepositoryStateCache.java:110)
	at org.eclipse.egit.ui.internal.decorators.DecoratableResourceAdapter.<init>(DecoratableResourceAdapter.java:75)
	at org.eclipse.egit.ui.internal.decorators.GitLightweightDecorator.decorateResource(GitLightweightDecorator.java:199)
	at org.eclipse.egit.ui.internal.decorators.GitLightweightDecorator.decorate(GitLightweightDecorator.java:172)
	at org.eclipse.ui.internal.decorators.LightweightDecoratorDefinition.decorate(LightweightDecoratorDefinition.java:250)
	at org.eclipse.ui.internal.decorators.LightweightDecoratorManager$LightweightRunnable.run(LightweightDecoratorManager.java:105)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.ui.internal.decorators.LightweightDecoratorManager.decorate(LightweightDecoratorManager.java:360)
	at org.eclipse.ui.internal.decorators.LightweightDecoratorManager.getDecorations(LightweightDecoratorManager.java:346)
	at org.eclipse.ui.internal.decorators.DecorationScheduler$1.queue(DecorationScheduler.java:419)
	at org.eclipse.ui.internal.decorators.DecorationScheduler$1.run(DecorationScheduler.java:397)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Comment 1 Matthias Sohn CLA 2022-03-07 06:09:15 EST
While decorating Eclipse resources in the UI EGit asks JGit to read the symbolic ref "HEAD" of some repository and fails doing that since some other process is using the file at that time and Windows refuses to read its content.

Maybe that's caused by some virus scanner running on your machine ?
Comment 2 Thomas Wolf CLA 2022-03-14 18:16:54 EDT
I fear this exception might be raised because the pull might be writing HEAD at the same time.
Comment 3 Thomas Wolf CLA 2022-03-14 20:08:48 EDT
See also [1]. It appears that atomic rename isn't really atomic on Windows/NTFS?

The exception message is misleading; the same message is also issued if the file is being written in the same process.

Judging from that, we either must have re-try loops (with a back-off, I suppose) wherever we try to read a file that may be concurrently written, or, if we trust that stackoverflow thread, use some NIO reading method. Probably re-try loops would be safer? (I.e., if we get a FileNotFoundException but then find the file existing and being a file, sleep for some delay and then re-try; increase delays slightly each time, and abort after some number of iterations.)

That might not only concern loose refs but also the packed refs file, or other files. (Git configs come to mind.)

[1] https://stackoverflow.com/questions/29923008/how-to-create-then-atomically-rename-file-in-java-on-windows
Comment 4 Eclipse Genie CLA 2022-03-15 14:29:47 EDT
New Gerrit change created: https://git.eclipse.org/r/c/jgit/jgit/+/191924
Comment 6 Matthias Sohn CLA 2022-03-19 18:11:46 EDT
(In reply to Thomas Wolf from comment #2)
> I fear this exception might be raised because the pull might be writing HEAD
> at the same time.

but aren't concurrent pulls always on different repositories ?
Comment 7 Thomas Wolf CLA 2022-03-20 05:56:31 EDT
(In reply to Matthias Sohn from comment #6)
> (In reply to Thomas Wolf from comment #2)
> > I fear this exception might be raised because the pull might be writing HEAD
> > at the same time.
> 
> but aren't concurrent pulls always on different repositories ?

Yes, but the re-decorations may not be. I think if you have projects from several repos in the package explorer and pull all repos, the index change or ref change from the first triggers re-decoration of all of them. So on projects from the other repos, we might run into this problem. Probably the same with the repositories view.

The change is merged in JGit and is available in EGit nightly; update site https://download.eclipse.org/egit/updates-nightly . I'm closing this now.

@Kenneth, feel free to re-open if it re-occurs with a JGit >= 6.2.0.202203192106.