git/list[1] front-page[2] threads[3] people[4] search[5] about
 

File locking issues with fetching submodules in parallel

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Feb 20, 2018, 16:20 UTC
Message-ID
<nycvar.QRO.7.76.6.1802201709570.31@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz>
Hi Stefan (and other submodule wizards),
Igor Melnichenko reported a submodule problem in a Git for Windows ticket:
	https://github.com/git-for-windows/git/issues/1469

Part of it is due to Windows' behavior where you cannot read the same file in one process while you write it in another process.

The other part is how our submodule code works in parallel. In particular, we seem to write the `.git` file maybe even every time a submodule is fetched, unconditionally, not even testing whether the .git file is already there and contains the correct contents?

For some reason, the bug reporter saw a "Permission denied" on the `.git` file when we try to write it (and I am pretty certain that I tracked it down correctly to the `connect_work_tree_and_git_dir()` function). The intermittent "Permission denied" error seems to suggest that *another* process is accessing this file while we are writing it.

It also seems that this problem becomes worse if the firewall is turned on, in which case a couple of network operations become a little slower (which I could imagine to be the factor making the problems more likely).

A plausible workaround would be to write the `.git` file only when needed (which also would fix a bug on macOS/Linux, although the window is substantially smaller: the reader could potentially read a partially-written file).

But maybe we are simply holding onto an open handle to the `.git` file too long?

I tried to put together a bit more information here:
https://github.com/git-for-windows/git/issues/1469#issuecomment-366932746

Do you think there is an easy solution for this? You're much deeper in the submodule code than I ever want to be...

Thanks, Dscho

Next: Stefan Beller
Message 1 of 3 in “File locking issues with fetching submodules in parallel”
  1. Johannes SchindelinFeb 20, 2018
  2. Stefan BellerFeb 20, 2018
  3. Johannes SchindelinFeb 20, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.