# 'git repack' and repack.writeBitmaps=true with kept packs 2 messages from 2016-12-01 to 2016-12-01. Participants: Steven Noonan, Jeff King. Thread: https://gitlist.dev/t/44577 ## Steven Noonan, 2016-12-01 00:15 Subject: 'git repack' and repack.writeBitmaps=true with kept packs Message-ID: URL: https://gitlist.dev/e/CAKbGBLjZ2WLVRM9f%3Dby337xLhPgKCy10T8ra6Qz7OWA%3DQF-5yA%40mail.gmail.com ``` I have some unexpected behavior with 'git repack' on git 2.10.2 and 2.11.0. $ cat /etc/gitconfig [pack] writeBitmapHashCache = true [repack] writeBitmaps = true $ touch objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.keep $ find objects objects objects/pack objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.keep objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.idx objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.bitmap objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.pack objects/info objects/info/packs $ git repack -Adfl Counting objects: 16321, done. Delta compression using up to 24 threads. Compressing objects: 100% (16118/16118), done. Writing objects: 100% (16321/16321), done. Reusing bitmaps: 110, done. Selecting bitmap commits: 3257, done. Building bitmaps: 100% (137/137), done. Total 16321 (delta 11568), reused 4507 (delta 0) $ du -sh objects/pack/pack* 100K objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.bitmap 524K objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.idx 512 objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.keep 4.3M objects/pack/pack-3841d81123a96cedeb3c1bd7acf7e29bfba26639.pack 100K objects/pack/pack-6043151bee7bd61bdae6e3a2ba0f13cd1b0277af.bitmap 524K objects/pack/pack-6043151bee7bd61bdae6e3a2ba0f13cd1b0277af.idx 5.8M objects/pack/pack-6043151bee7bd61bdae6e3a2ba0f13cd1b0277af.pack It seems like it's behaving as though I've provided --pack-kept-objects. In order to ensure the .bitmap is created, it repacks everything, including everything in existing .pack files (not respecting .keep). But then it's not deleting the old .pack file (oddly, respecting .keep). What I'd expect it to do here is ignore the 'repack.writeBitmaps = true' value if there's a .keep that needs to be respected. Is this not a correct assumption? ``` ## Jeff King, 2016-12-01 05:16 Subject: Re: 'git repack' and repack.writeBitmaps=true with kept packs Message-ID: <20161201051640.gsavaexw55mwycza@sigill.intra.peff.net> URL: https://gitlist.dev/e/20161201051640.gsavaexw55mwycza%40sigill.intra.peff.net In-Reply-To: ``` On Wed, Nov 30, 2016 at 04:15:33PM -0800, Steven Noonan wrote: > It seems like it's behaving as though I've provided > --pack-kept-objects. In order to ensure the .bitmap is created, it > repacks everything, including everything in existing .pack files (not > respecting .keep). But then it's not deleting the old .pack file > (oddly, respecting .keep). Right, that's exactly what's happening. The bitmaps require a completely reachable set inside the pack, so if you omit some objects that are in .keep packs, we cannot generate the bitmap. So we have to either disable bitmaps, or pack the kept objects. By default, we do the latter (and I'll explain why in a minute). We can't delete the .keep packfiles because we don't know for sure that we've included all of their contents in the new pack (not to mention that somebody asked to keep them, and we don't know why; we should respect that). > What I'd expect it to do here is ignore the 'repack.writeBitmaps = > true' value if there's a .keep that needs to be respected. Is this not > a correct assumption? In practice, I think that ends up worse. The .keep files are used by receive-pack as lockfiles for incoming pushes. So imagine you kick off a full repack just as somebody is pushing, and repack sees the temporary .keep file. Your options are: 1. Disable bitmaps, leaving the repository with no bitmaps at all until the next repack (because you're deleting the old bitmaps along with the old, non-kept pack). 2. Duplicate the newly pushed objects in the pack (if they're even reachable; you're also racing to see the ref updates). Now you have bitmaps, but you're wasting a bit of space to store the racy push twice (and it goes away next time you repack). If you're running a Git server which depends on bitmaps for good performance, then (2) is much better. And that's the default. If you want to override it, you can pass --no-pack-kept-objects, or set repack.packKeptObjects to false. I think the documentation for --pack-kept-objects could be a bit more clear for this case. It doesn't mention the default value, nor that what you really want with "-b" is probably "--no-pack-kept-objects". -Peff ```