git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/1] bundle: cleanup lock files on error

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Nov 15, 2018, 16:32 UTC
Message-ID
<nycvar.QRO.7.76.6.1811151725310.41@tvgsbejvaqbjf.bet>
In-Reply-To
<20181115133749.GA26164@sigill.intra.peff.net>
Hi Peff,
On Thu, 15 Nov 2018, Jeff King wrote:
Show 61 quoted lines
> On Thu, Nov 15, 2018 at 01:57:45PM +0100, Johannes Schindelin wrote:
> 
> > > I looked at the test to see if it would pass, but it is not even
> > > checking anything about lockfiles! It just checks that we exit 1 by
> > > returning up the callstack instead of calling die(). And of course it
> > > would not have a problem under Linux either way. But if I run something
> > > similar under strace, I see:
> > > 
> > >   $ strace ./git bundle create foobar.bundle HEAD..HEAD
> > >   [...]
> > >   openat(AT_FDCWD, "/home/peff/compile/git/foobar.bundle.lock", O_RDWR|O_CREAT|O_EXCL|O_CLOEXEC, 0666) = 3
> > >   [...]
> > >   close(3)                                = 0
> > >   unlink("/home/peff/compile/git/foobar.bundle.lock") = 0
> > >   exit_group(128)                         = ?
> > > 
> > > which seems right.
> > 
> > Without the fix, the added regression test fails thusly:
> > 
> > -- snip --
> > [...]
> > ++ test_expect_code 1 git bundle create foobar.bundle master..master
> > ++ want_code=1
> > ++ shift
> > ++ git bundle create foobar.bundle master..master
> > fatal: Refusing to create empty bundle.
> > warning: unable to unlink 'C:/git-sdk-64/usr/src/git/wip2/t/trash directory.t5607-clone-bundle/foobar.bundle.lock': Permission denied
> 
> Hmph. So who has it open, and why isn't the tempfile code working as
> designed?
> 
> Aha, I see the problem. We dup() the descriptor in create_bundle(). So
> the patch _is_ necessary and (fairly) correct. But the explanation
> probably ought to be something like:
> 
>   In create_bundle(), we duplicate the lockfile descriptor via dup().
>   This means that even though the lockfile code carefully calls close()
>   before unlinking the lockfile, we may still have the file open. Unix
>   systems don't care, but under Windows, this prevents the unlink
>   (causing an annoying warning and a stale lockfile).
> 
> But that also means that all of the other places we could die (e.g., in
> write_or_die) are going to have the same problem. We've fixed only one.
> Is there a way we can avoid doing the dup() in the first place?
> 
> The comment there explains that we duplicate because write_pack_data()
> will close the descriptor. Unfortunately, that's hard to change because
> it comes from run-command. But we don't actually need the descriptor
> ourselves after it's closed; we're just trying to appease the lockfile
> code; see e54c347c1c (create_bundle(): duplicate file descriptor to
> avoid closing it twice, 2015-08-10).
> 
> We just need some reasonable way of telling the lock code what's
> happening. Something like the patch below, which is a moral revert of
> e54c347c1c, but instead wrapping the "lock->fd = -1" in an official API.
> 
> Does this make your warning go away?
> 
> diff --git a/bundle.c b/bundle.c
> [...]

I cannot claim that I wrapped my head around your explanation or your diff (I am busy trying to prepare Git for Windows' `master` for rebasing to v2.20.0-rc0), but it does fix the problem. Thank you so much!

The line `test_expect_code 1 ...` needs to be adjusted to `test_expect_code 128`, of course, and to discern from the fixed problem (which also exits with code 128), the error output should be verified, like so:

-- snip --
test_expect_success 'try to create a bundle with empty ref count' '
	test_must_fail git bundle create foobar.bundle master..master 2>err &&
	test_i18ngrep "Refusing to create empty bundle" err
'
-- snap --

Do you want to integrate this test into your patch and run with it, or do you want me to shepherd your patch?

Ciao, Dscho

Previous: Jeff KingNext: Jeff King
Message 12 of 20 in “bundle: fix issue when bundles would be empty”
  1. 0/1 bundle: fix issue when bundles would be emptyJohannes Schindelin via GitGitGadget, Nov 13, 2018
  2. 1/1 bundle: refuse to create empty bundleGaël Lhez via GitGitGadget, Nov 13, 2018
  3. Stefan BellerNov 13, 2018
  4. Gaël LhezNov 13, 2018
  5. 0/1 bundle: fix issue when bundles would be emptyJohannes Schindelin via GitGitGadget, Nov 14, 2018
  6. 1/1 bundle: cleanup lock files on errorGaël Lhez via GitGitGadget, Nov 14, 2018
  7. Martin ÅgrenNov 14, 2018
  8. Stefan BellerNov 14, 2018
  9. Jeff KingNov 15, 2018
  10. Johannes SchindelinNov 15, 2018
  11. Jeff KingNov 15, 2018
  12. Johannes SchindelinNov 15, 2018
  13. Jeff KingNov 15, 2018
  14. Johannes SchindelinNov 15, 2018
  15. bundle: dup() output descriptor closer to point-of-useJeff King, Nov 16, 2018
  16. Johannes SchindelinNov 16, 2018
  17. Junio C HamanoNov 17, 2018
  18. Duy NguyenNov 15, 2018
  19. Stefan BellerNov 13, 2018
  20. Johannes SchindelinNov 14, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.