git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.

From
Tao Klerks <tao@klerks.biz>
Date
Sep 4, 2023, 14:41 UTC
Message-ID
<CAPMMpoixKnr4BkKd8jeU+79Edhqtu4R7m8=BX4ZSYKdBHDzK=w@mail.gmail.com>
Hi folks,

I work with a project that often, or typically, benefits from having multiple worktrees where users are working with different versions of the code.

Worktrees make particular sense in this project for all the usual reasons, plus the fact that it is a very *large* repo - multiple gigs of repo, and very fast-moving; cloning and fetching less often is nice.

One problem that we found users to have, early on when we introduced worktrees, was that it was not obvious to users that there was (or why there was) a "main" worktree, containing the actual ".git" repo, and "subsidiary" worktrees, in the same directory location. Git by default makes worktrees *subfolders* of the initial/main worktree/folder, but we put them alongside each other to make other project tooling be able to treat all worktrees consistently; in daily use there is absolutely no difference between them, in all the "project tooling" there cannot be a practical difference... but if you choose to delete a worktree, and it happens to be the "special" one that contains the repo... you've just lost stuff that you didn't expect to lose (any local branches and stashes, basically).

Because worktree use was so useful/widespread/critical on this project, and we already had a custom cloning process that introduced selective refspecs etc, we introduced a special clone topology: the initial clone is a bare repo, and that folder gets a specific clear name (ending in .git). Then we create worktrees attached to that bare repo.

Generally speaking, this has worked *very* well: I would recommend it generally as a recognized/supported local-repo-setup. The most important thing that makes this *possible* is the fact that "git rev-parse --is-bare-repository" returns True in the bare repo folder, where the lack of index and HEAD shouldn't bother git, and it returns False in any one of the worktrees. It feels like things were designed to work this way, even though I can find no explicit mention of this topology in the docs.

However, from time to time something weird happens: Today I finally started to understand why I was seeing a crazy GC error about bitmaps, intermittently: It seems to be because "git gc" wants to create bitmaps in bare repos, but can't do so properly when we're in a partial clone... or something like that?

EG repro: ``` git clone --bare https://github.com/ksylor/ohshitgit dangit_shared.git --filter=blob:none git -C dangit_shared.git worktree add $PWD/dangit_wt3 cd dangit_wt3/ echo "this is some new unique blob in the repo" > new_blob.txt git add new_blob.txt && git commit -m "new blob" cd ../dangit_shared.git/ git gc ```

This yields, at the end of the GC run: ``` warning: Failed to write bitmap index. Packfile doesn't have full closure (object bf86ed1b2602ac3a8d4724bcdf6707b156673aac is missing) fatal: failed to write bitmap index fatal: failed to run repack ```

On the other hand, running "git gc" in one of the worktrees works fine (except you first need to delete a couple of ".tmp-*" files from the "objects/pack" folder, if you already got the error above).

I at first thought this was a bug - but as I realized the problematic behavior was tied to the "core.bare" setting (and its expression at runtime through "git rev-parse --is-bare-repository"), it became more obvious that maybe the system could/should be able to make assumptions about the kind of repo that has this "true", and assume there are no local-object-derived non-promisor packfiles (or whatever it is about this example that makes things unhappy).

So, I guess I'm confused as to what "core.bare" is supposed to mean: Is it intended to mean "there is no index nor HEAD here, and that's good, don't worry" (in which case my setup is presumably "supported", and the gc behavior is buggy?), or is it intended to mean "this is the kind of repository in which there are no worktrees" (in which case I am abusing the system and get the errors I deserve)?

CC Dscho, who made the worktree support that I rely on work about 16 years ago it seems, and Taylor Blau and Patrick Steinhardt, who I think have made changes in the area of the code concerned with whether or not we try to create bitmaps, and might have an opinion as to whether the assumptions made by "gc" at the moment are unsafe, or my use or "core.bare" in this context is wrong.

Thanks for any feedback! Tao

Next: Tao Klerks
Message 1 of 31 in “Is "bare"ness in the context of multiple worktrees weird? Bitmap error in git gc.”
  1. Tao KlerksSep 4, 2023
  2. Tao KlerksSep 4, 2023
  3. Tao KlerksSep 4, 2023
  4. Kristoffer HaugsbakkSep 4, 2023
  5. Kristoffer HaugsbakkSep 4, 2023
  6. Eric SunshineSep 5, 2023
  7. Kristoffer HaugsbakkSep 6, 2023
  8. Sergey OrganovSep 6, 2023
  9. Kristoffer HaugsbakkSep 6, 2023
  10. Tao KlerksSep 6, 2023
  11. Junio C HamanoSep 6, 2023
  12. Sergey OrganovSep 6, 2023
  13. Junio C HamanoSep 6, 2023
  14. Sergey OrganovSep 6, 2023
  15. Tao KlerksSep 7, 2023
  16. Sergey OrganovSep 7, 2023
  17. Kristoffer HaugsbakkSep 7, 2023
  18. Kristoffer HaugsbakkSep 7, 2023
  19. Junio C HamanoSep 7, 2023
  20. Junio C HamanoSep 6, 2023
  21. Kristoffer HaugsbakkSep 6, 2023
  22. Junio C HamanoSep 6, 2023
  23. Sergey OrganovSep 6, 2023
  24. Tao KlerksSep 5, 2023
  25. Eric SunshineSep 5, 2023
  26. Junio C HamanoSep 5, 2023
  27. Eric SunshineSep 5, 2023
  28. Junio C HamanoSep 5, 2023
  29. Tao KlerksSep 5, 2023
  30. Kristoffer HaugsbakkSep 6, 2023
  31. Tao KlerksSep 5, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.