Bug 577358 - Update to Git 2.34.0 freezes application that uses JGit
Summary: Update to Git 2.34.0 freezes application that uses JGit
Status: RESOLVED FIXED
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: 5.10   Edit
Hardware: All All
: P3 blocker (vote)
Target Milestone: 6.0   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-11-19 04:23 EST by Alexander Veit CLA
Modified: 2021-11-26 10:29 EST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Veit CLA 2021-11-19 04:23:51 EST
After an update to the latest Git version (2.25.1 to 2.34.0) the application freezes due to a call to JGit, most likely org.eclipse.jgit.util.FS.discoverGitSystemConfig() called by the FS_POSIX class.

The exact call site could not be determined because the error stops all threads in the calling JVM and cannot be reproduced under the Eclipse debugger.

Strace suggests that the JVM receives a SIGTTOU from the child process "git config --system --edit" which is created as a "trick" in order to determine the global gitconfig file.


Environment:
OS: Linux Mint 20
Kernel: 5.4.0-90-generic
Git 2.34.0 from http://ppa.launchpad.net/git-core/ppa/ubuntu
Comment 1 Andrey Loskutov CLA 2021-11-19 04:33:09 EST
Can you create thread dump with jstack please?
Comment 2 Thomas Wolf CLA 2021-11-19 04:43:10 EST
Probably caused by this:

https://github.com/git/git/commit/3d411afabc9a96f41d47c07d6af6edda3d29ec92#diff-01b59b6a71e42b9c1251ffbf76a1119b965be087a78538e80e01f9239c8e5880

IMO this is a bug in C git; this breaks our use case completely.
Comment 3 Alexander Veit CLA 2021-11-19 06:22:01 EST
Andrey, jstack does not work. The JVM is completeley locked down.

./jstack 15882
15882: Unable to open socket file /proc/15882/root/tmp/.java_pid15882: target process 15882 doesn't respond within 10500ms or HotSpot VM not loaded

The Java version is

./java --version
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)
Comment 4 Alexander Veit CLA 2021-11-19 06:35:12 EST
Manually sending SIGCONT to the JVM process and SIGKILLing the git child process lets the application resume its work.

The JVM spits out the following stack trace on stderr:

[...]
"AWT-EventQueue-0" #15 prio=6 os_prio=0 cpu=8944.53ms elapsed=957.58s tid=0x00007f19b8b34000 nid=0x3e24 in Object.wait()  [0x00007f194736000
0]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(java.base@11.0.10/Native Method)
        - waiting on <no object reference available>
        at java.lang.Object.wait(java.base@11.0.10/Object.java:328)
        at java.lang.ProcessImpl.waitFor(java.base@11.0.10/ProcessImpl.java:495)
        - waiting to re-lock in wait() <0x00000000ccbd2078> (a java.lang.ProcessImpl)
        at org.eclipse.jgit.util.FS.readPipe(FS.java:1374)
        at org.eclipse.jgit.util.FS.discoverGitSystemConfig(FS.java:1520)
        at org.eclipse.jgit.util.FS.getGitSystemConfig(FS.java:1544)
        at org.eclipse.jgit.util.SystemReader$Default.openSystemConfig(SystemReader.java:86)
        at org.eclipse.jgit.util.SystemReader.getSystemConfig(SystemReader.java:363)
        at org.eclipse.jgit.util.SystemReader.getUserConfig(SystemReader.java:311)
        at org.eclipse.jgit.internal.storage.file.FileRepository.<init>(FileRepository.java:162)
        at org.eclipse.jgit.lib.BaseRepositoryBuilder.build(BaseRepositoryBuilder.java:583)
        at org.eclipse.jgit.api.Git.open(Git.java:93)
        at org.eclipse.jgit.api.Git.open(Git.java:73)
[...]
Comment 5 Thomas Wolf CLA 2021-11-19 08:08:01 EST
I really think this needs to be fixed in C git. JGit uses the way to find the location of the git system config via

  GIT_EDITOR=echo git --system --edit

which is also the recommended way per [1].

The C git commit linked in comment 2 breaks this if executed from Java since there is no terminal, and C git unconditionally attempts terminal operations even when there is no terminal.

[1] https://yhbt.net/lore/all/nycvar.QRO.7.76.6.1903221436590.41@tvgsbejvaqbjf.bet/T/
Comment 6 Thomas Wolf CLA 2021-11-19 11:52:13 EST
Of course I meant

  GIT_EDITOR=echo git config --system --edit

Where's the bug tracker of C git?? If it's the mailing list, can someone report this there, please?

If they really don't want to support this GIT_EDIT=echo approach anymore, then git config *needs* to get a --show-files option, producing output on stdout, for instance like

$ git config --show-files
local=/Some/path/somewhere/.git/config
global=/home/user/.gitconfig
system=/etc/.gitconfig

and if invoked with -z, NUL-terminated and the keys ("local", "global", "system", etc.) not localized but always exactly as shown. Then we could use the GIT_EDITOR approach for versions < 2.34.0 and the new command for later versions. "worktree" to be shown only if it exists, "local" shown only if invoked in a git directory. Order of files from most specific to least specific or vice versa, but in any case defined. All paths absolute.
Comment 7 Thomas Wolf CLA 2021-11-19 12:07:40 EST
Or we have to parse the output of

  git config --system --show-origin --list
Comment 8 Eclipse Genie CLA 2021-11-19 13:22:18 EST
New Gerrit change created: https://git.eclipse.org/r/c/jgit/jgit/+/187938
Comment 9 Alexander Veit CLA 2021-11-22 05:44:28 EST
Reported the issue to upstream mailing list git@vger.kernel.org (https://marc.info/?l=git&m=163757055317893&w=2).
Comment 11 Thomas Wolf CLA 2021-11-23 06:31:51 EST
(In reply to Alexander Veit from comment #9)
> Reported the issue to upstream mailing list git@vger.kernel.org
> (https://marc.info/?l=git&m=163757055317893&w=2).

Thanks for reporting it. It's on the radar for being fixed in 2.34.1. Unfortunately we still need our work-around -- we can't have applications hang whenever some user out there has git 2.34.0 installed.

Would still be great if C git had an official standard API to get these config file locations. Both ways to get the info currently are kludges.